# Derandomization via small sample spaces

## Abstract

Many randomized algorithms run successfully even when the random choices they utilize are not fully independent. For the analysis some limited amount of independence, like *k-wise independence* for some fixed *k*, often suffices. In these cases, it is possible to replace the appropriate exponentially large sample spaces required to simulate all random choices of the algorithms by ones of polynomial size. This enables one to *derandomize* the algorithms, that is, convert them into deterministic ones, by searching the relatively small sample spaces deterministically. If a random variable attains a certain value with positive probability, then we can actually search and find a point in which it attains such a value.

The observation that *n* − 1 *pairwise independent* nontrivial random variables can be defined over a sample space of size *n* has been mentioned already long ago, see [11], [23]. The pairwise independent case has been a crucial ingredient in the construction of efficient hashing schemes in [14], [17]. A more general construction, of small sample spaces supporting *k*-wise independent random variables, appeared in [19]. For the case of binary, uniform random variables this is treated under the name *orthogonal arrays* in the Coding Theory literature, see, e.g., [27]. Most constructions are based on some simple properties of polynomials over a finite field or on certain explicit error correcting codes.

Several researchers realized that constructions of this type are useful for derandomizing *parallel* algorithms, since one may simply check all points of the sample space in parallel. Papers pursuing this idea include [1], [22], [24], and papers dealing with the properties of the constructions in which the sample spaces are not necessarily uniform include [20], [21]. It can be shown that for fixed *k*, the minimum size of a sample space supporting *n**k*-wise independent random variables is *Ω(n*^{⌊k/2⌋}. For the binary uniform case this is essentially the Rao bound [30] (see also [12], [16]), whereas for the general case it is shown in

The above techniques have been applied in numerous papers dealing with derandomization, and we make no attempt to list all of them here. Examples include parallelization of derandomized geometric algorithms in [10], [18], and various parallel graph algorithms [1], [9], [22], [24], [28]. It turned out that some variants of the techniques are also useful in derandomizing *sequential* algorithms

In the talk I will survey the basic ideas in the constructions of small sample spaces and discuss some of the applications, focusing on various recent results that illustrate the somewhat surprising relevance of the techniques to the solutions of several algorithmic problems.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.N. Alon, L. Babai and A. Itai. A fast and simple randomized parallel algorithm for the maximal independent set problem.
*J. Alg.*, 7:567–583, 1986.Google Scholar - 2.N. Alon, J. Bruck, J. Naor, M. Naor and R. Roth. Construction of asymptotically good, low-rate error-correcting codes through pseudo-random graphs.
*IEEE Trans. Info. Theory*, 38:509–516, 1992.Google Scholar - 3.N. Alon, O. Goldreich, J. Håstad and R. Peralta. Simple constructions of almost
*k*-wise independent random variables.*Random Structures and Algorithms*, 3(3):289–303, 1992.Google Scholar - 4.N. Alon, Y. Matias and M. Szegedy. The space complexity of approximating the frequency moments. In
*Proc. of the 28th ACM Symp. on Theory of Computing*, 1996, in press.Google Scholar - 5.N. Alon and M. Naor. Derandomization, witnesses for Boolean matrix multiplication and construction of perfect hash functions. To appear in
*Algorithmica*.Google Scholar - 6.N. Alon and J. H. Spencer.
*The Probabilistic Method*. Wiley, 1992.Google Scholar - 7.N. Alon, R. Yuster and U. Zwick. Color-coding.
*J. ACM*42:844–856, 1995.Google Scholar - 8.Y. Azar, R. Motwani and J. Naor. Approximating arbitrary probability distributions using small sample spaces. Manuscript, 1990.Google Scholar
- 9.B. Berger and J. Rompel. Simulating (log
^{c}*n*)-wise independence in NC.*Journal of the ACM*, 38:1026–1046, 1991.Google Scholar - 10.B. Berger, J. Rompel and P. W. Shor. Efficient NC algorithms for set cover with applications to learning and geometry. In
*Proc. 30th IEEE Symposium on Foundations of Computer Science*, pages 54–59, 1989.Google Scholar - 11.
- 12.B. Chor, O. Goldreich, J. Hastad, J. Friedman, S. Rudich and R. Smolensky. The Bit Extraction Problem or
*t*-Resilient Functions. In 26th*Annual Symposium on Foundations of Computer Science, Portland, Oregon*, pages 396–407, 1985.Google Scholar - 13.S. Chari, P. Rohatgi and A. Srinivasan. Improved algorithms via approximations of probability distributions. In
*Proc. 26th ACM Symposium on Theory of Computing*, pages 584–592, 1994.Google Scholar - 14.L. Carter and M. Wegman. Universal classes of Hash functions.
*J. Computer System Sciences*, 18:143–154, 1979.Google Scholar - 15.G. Even, O. Goldreich, M. Luby, N. Nisan and B. Velićković. Approximations of general independent distributions. In
*Proc. 24th ACM Symposium on Theory of Computing*, pages 10–16, 1992.Google Scholar - 16.J. Friedman. On the bit extraction problem. In
*Proc. 33rd IEEE Symposium on Foundations of Computer Science*, pages 314–319, 1992.Google Scholar - 17.M. Fredman, J. Komlos and E. Szemerédi. Storing a sparse table with
*O*(1) worst-case access time. In*Proc. 23rd IEEE Symposium on Foundations of Computer Science*, pages 165–169, 1982.Google Scholar - 18.M. T. Goodrich. Geometric partitioning made easier, even in parallel. In
*Proc. 9th ACM Symp. Comput. Geom.*, pages 73–82, 1993.Google Scholar - 19.A. Joffe. On a set of almost deterministic
*k*-independent random variables.*Annals of Probability*, 2:161–162, 1974.Google Scholar - 20.D. Koller and N. Megiddo, Constructing small sample spaces satisfying given constraints. In
*Proc. of the 25th Annual ACM Symposium on Theory of Computing*, pages 268–277, 1993.Google Scholar - 21.H. Karloff and Y. Mansour. On construction of
*k*-wise independent random variables. In*Proc. of the 26th Annual ACM Symposium on Theory of Computing*, pages 564–573, 1994.Google Scholar - 22.R. Karp and A. Wigderson. A fast parallel algorithm for the maximum independent set problem.
*J. ACM*, 32: 762–773, 1985.Google Scholar - 23.H. O. Lancaster. Pairwise statistical independence.
*Ann. Math. Stat.*36:1313–1317, 1965.Google Scholar - 24.M. Luby. A simple parallel algorithm for the maximal independent set problem.
*SIAM J. Comput.*, 15(4):1036–1053, 1986.CrossRefGoogle Scholar - 25.M. Luby. Removing randomness in parallel computation without a processor penalty.
*J. Comput. Syst. Sci.*, 47(2):250–286, 1993.Google Scholar - 26.R. Motwani, J. Naor and M. Naor. The probabilistic method yields deterministic parallel algorithms.
*J. Comput. Syst. Sci.*, 49:478–516, 1994.Google Scholar - 27.F. J. MacWilliams and N. J. A. Sloane.
*The Theory of Error-Correcting Codes*. North Holland, Amsterdam, 1977.Google Scholar - 28.J. Naor and M. Naor. Small-bias probability spaces: efficient constructions and applications.
*SIAM J. Comput.*, 22(4):838–856, 1993.Google Scholar - 29.P. Raghavan. Probabilistic construction of deterministic algorithms: approximating packing integer programs.
*J. Comput. Syst. Sci.*, 37:130–143, 1988.Google Scholar - 30.C. R. Rao. Factorial experiments derivable from combinatorial arrangements of arrays.
*J. Royal Stat. Soc.*9: 128–139, 1947.Google Scholar - 31.L. J. Schulman. Sample spaces uniform on neighborhoods. In
*Proceedings of the*24*th Annual ACM Symposium on Theory of Computing*, pages 17–25, 1992.Google Scholar - 32.J. Spencer.
*Ten Lectures on the Probabilistic Method*. SIAM, 1987.Google Scholar