Derandomization via small sample spaces

  • Noga Alon
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1097)


Many randomized algorithms run successfully even when the random choices they utilize are not fully independent. For the analysis some limited amount of independence, like k-wise independence for some fixed k, often suffices. In these cases, it is possible to replace the appropriate exponentially large sample spaces required to simulate all random choices of the algorithms by ones of polynomial size. This enables one to derandomize the algorithms, that is, convert them into deterministic ones, by searching the relatively small sample spaces deterministically. If a random variable attains a certain value with positive probability, then we can actually search and find a point in which it attains such a value.

The observation that n − 1 pairwise independent nontrivial random variables can be defined over a sample space of size n has been mentioned already long ago, see [11], [23]. The pairwise independent case has been a crucial ingredient in the construction of efficient hashing schemes in [14], [17]. A more general construction, of small sample spaces supporting k-wise independent random variables, appeared in [19]. For the case of binary, uniform random variables this is treated under the name orthogonal arrays in the Coding Theory literature, see, e.g., [27]. Most constructions are based on some simple properties of polynomials over a finite field or on certain explicit error correcting codes.

Several researchers realized that constructions of this type are useful for derandomizing parallel algorithms, since one may simply check all points of the sample space in parallel. Papers pursuing this idea include [1], [22], [24], and papers dealing with the properties of the constructions in which the sample spaces are not necessarily uniform include [20], [21]. It can be shown that for fixed k, the minimum size of a sample space supporting nk-wise independent random variables is Ω(nk/2⌋. For the binary uniform case this is essentially the Rao bound [30] (see also [12], [16]), whereas for the general case it is shown in

The above techniques have been applied in numerous papers dealing with derandomization, and we make no attempt to list all of them here. Examples include parallelization of derandomized geometric algorithms in [10], [18], and various parallel graph algorithms [1], [9], [22], [24], [28]. It turned out that some variants of the techniques are also useful in derandomizing sequential algorithms

In the talk I will survey the basic ideas in the constructions of small sample spaces and discuss some of the applications, focusing on various recent results that illustrate the somewhat surprising relevance of the techniques to the solutions of several algorithmic problems.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    N. Alon, L. Babai and A. Itai. A fast and simple randomized parallel algorithm for the maximal independent set problem. J. Alg., 7:567–583, 1986.Google Scholar
  2. 2.
    N. Alon, J. Bruck, J. Naor, M. Naor and R. Roth. Construction of asymptotically good, low-rate error-correcting codes through pseudo-random graphs. IEEE Trans. Info. Theory, 38:509–516, 1992.Google Scholar
  3. 3.
    N. Alon, O. Goldreich, J. Håstad and R. Peralta. Simple constructions of almost k-wise independent random variables. Random Structures and Algorithms, 3(3):289–303, 1992.Google Scholar
  4. 4.
    N. Alon, Y. Matias and M. Szegedy. The space complexity of approximating the frequency moments. In Proc. of the 28th ACM Symp. on Theory of Computing, 1996, in press.Google Scholar
  5. 5.
    N. Alon and M. Naor. Derandomization, witnesses for Boolean matrix multiplication and construction of perfect hash functions. To appear in Algorithmica.Google Scholar
  6. 6.
    N. Alon and J. H. Spencer. The Probabilistic Method. Wiley, 1992.Google Scholar
  7. 7.
    N. Alon, R. Yuster and U. Zwick. Color-coding. J. ACM 42:844–856, 1995.Google Scholar
  8. 8.
    Y. Azar, R. Motwani and J. Naor. Approximating arbitrary probability distributions using small sample spaces. Manuscript, 1990.Google Scholar
  9. 9.
    B. Berger and J. Rompel. Simulating (logc n)-wise independence in NC. Journal of the ACM, 38:1026–1046, 1991.Google Scholar
  10. 10.
    B. Berger, J. Rompel and P. W. Shor. Efficient NC algorithms for set cover with applications to learning and geometry. In Proc. 30th IEEE Symposium on Foundations of Computer Science, pages 54–59, 1989.Google Scholar
  11. 11.
    S. Bernstein. Theory of Probability (3rd Edition). GTTI, Moscow, 1945.Google Scholar
  12. 12.
    B. Chor, O. Goldreich, J. Hastad, J. Friedman, S. Rudich and R. Smolensky. The Bit Extraction Problem or t-Resilient Functions. In 26th Annual Symposium on Foundations of Computer Science, Portland, Oregon, pages 396–407, 1985.Google Scholar
  13. 13.
    S. Chari, P. Rohatgi and A. Srinivasan. Improved algorithms via approximations of probability distributions. In Proc. 26th ACM Symposium on Theory of Computing, pages 584–592, 1994.Google Scholar
  14. 14.
    L. Carter and M. Wegman. Universal classes of Hash functions. J. Computer System Sciences, 18:143–154, 1979.Google Scholar
  15. 15.
    G. Even, O. Goldreich, M. Luby, N. Nisan and B. Velićković. Approximations of general independent distributions. In Proc. 24th ACM Symposium on Theory of Computing, pages 10–16, 1992.Google Scholar
  16. 16.
    J. Friedman. On the bit extraction problem. In Proc. 33rd IEEE Symposium on Foundations of Computer Science, pages 314–319, 1992.Google Scholar
  17. 17.
    M. Fredman, J. Komlos and E. Szemerédi. Storing a sparse table with O(1) worst-case access time. In Proc. 23rd IEEE Symposium on Foundations of Computer Science, pages 165–169, 1982.Google Scholar
  18. 18.
    M. T. Goodrich. Geometric partitioning made easier, even in parallel. In Proc. 9th ACM Symp. Comput. Geom., pages 73–82, 1993.Google Scholar
  19. 19.
    A. Joffe. On a set of almost deterministic k-independent random variables. Annals of Probability, 2:161–162, 1974.Google Scholar
  20. 20.
    D. Koller and N. Megiddo, Constructing small sample spaces satisfying given constraints. In Proc. of the 25th Annual ACM Symposium on Theory of Computing, pages 268–277, 1993.Google Scholar
  21. 21.
    H. Karloff and Y. Mansour. On construction of k-wise independent random variables. In Proc. of the 26th Annual ACM Symposium on Theory of Computing, pages 564–573, 1994.Google Scholar
  22. 22.
    R. Karp and A. Wigderson. A fast parallel algorithm for the maximum independent set problem. J. ACM, 32: 762–773, 1985.Google Scholar
  23. 23.
    H. O. Lancaster. Pairwise statistical independence. Ann. Math. Stat. 36:1313–1317, 1965.Google Scholar
  24. 24.
    M. Luby. A simple parallel algorithm for the maximal independent set problem. SIAM J. Comput., 15(4):1036–1053, 1986.CrossRefGoogle Scholar
  25. 25.
    M. Luby. Removing randomness in parallel computation without a processor penalty. J. Comput. Syst. Sci., 47(2):250–286, 1993.Google Scholar
  26. 26.
    R. Motwani, J. Naor and M. Naor. The probabilistic method yields deterministic parallel algorithms. J. Comput. Syst. Sci., 49:478–516, 1994.Google Scholar
  27. 27.
    F. J. MacWilliams and N. J. A. Sloane. The Theory of Error-Correcting Codes. North Holland, Amsterdam, 1977.Google Scholar
  28. 28.
    J. Naor and M. Naor. Small-bias probability spaces: efficient constructions and applications. SIAM J. Comput., 22(4):838–856, 1993.Google Scholar
  29. 29.
    P. Raghavan. Probabilistic construction of deterministic algorithms: approximating packing integer programs. J. Comput. Syst. Sci., 37:130–143, 1988.Google Scholar
  30. 30.
    C. R. Rao. Factorial experiments derivable from combinatorial arrangements of arrays. J. Royal Stat. Soc. 9: 128–139, 1947.Google Scholar
  31. 31.
    L. J. Schulman. Sample spaces uniform on neighborhoods. In Proceedings of the 24th Annual ACM Symposium on Theory of Computing, pages 17–25, 1992.Google Scholar
  32. 32.
    J. Spencer. Ten Lectures on the Probabilistic Method. SIAM, 1987.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • Noga Alon
    • 1
  1. 1.School of Mathematical Sciences, Raymond and Beverly Sackler Faculty of Exact SciencesTel Aviv UniversityTel AvivIsrael

Personalised recommendations