Independence of Tabulation-Based Hash Classes

Abstract

A tabulation-based hash function maps a key into multiple derived characters which index random values in tables that are then combined with bitwise exclusive or operations to give the hashed value. Thorup and Zhang [9] presented tabulation-based hash classes that use linear maps over finite fields to map keys of the form (a,b) (composed of two characters, a and b, of equal length) to d derived characters in order to achieve d-wise independence. We present a variant in which d derived characters a + b·i, for i = 0,…,d − 1 (where arithmetic is over integers) are shown to yield (2d − 1)-wise independence. Thus to achieve guaranteed k-wise independence for k ≥ 6, our method reduces by about half the number of probes needed into the tables compared to Thorup and Zhang (they presented a different specialized scheme to give 4-wise [9] and 5-wise [10] independence).

Our analysis is based on an algebraic property that characterizes k-wise independence of tabulation-based hashing schemes, and combines this characterization with a geometric argument. We also prove a non-trivial lower bound on the number of derived characters necessary for k-wise independence with our and related hash classes.