Abstract
A Bloom filter is a very compact data structure that supports approximate membership queries on a set, allowing false positives.
We propose several new variants of Bloom filters and replacements with similar functionality. All of them have a better cache-efficiency and need less hash bits than regular Bloom filters. Some use SIMD functionality, while the others provide an even better space efficiency. As a consequence, we get a more flexible trade-off between false positive rate, space-efficiency, cache-efficiency, hash-efficiency, and computational effort. We analyze the efficiency of Bloom filters and the proposed replacements in detail, in terms of the false positive rate, the number of expected cache-misses, and the number of required hash bits. We also describe and experimentally evaluate the performance of highly-tuned implementations. For many settings, our alternatives perform better than the methods proposed so far.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bloom, B.H.: Space-time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7) (1970)
Broder, A., Mitzenmacher, M.: Network applications of bloom filters: A survey. Internet Mathematics, 1(4) (2004)
Cleary, J.G.: Compact hash tables using bidirectional linear probing. IEEE Transactions on Computers 33(9), 828–834 (1984)
Dillinger, P.C., Manolios, P.: Bloom filters in probabilistic verification. In: Hu, A.J., Martin, A.K. (eds.) FMCAD 2004. LNCS, vol. 3312, pp. 367–381. Springer, Heidelberg (2004)
Dillinger, P.C., Manolios, P.: Fast and accurate bitstate verification for SPIN. In: Graf, S., Mounier, L. (eds.) Model Checking Software. LNCS, vol. 2989, pp. 57–75. Springer, Heidelberg (2004)
Fan, L., Cao, P., Almeida, J.M., Broder, A.Z.: Summary cache: a scalable wide-area web cache sharing protocol. IEEE/ACM TON 8(3), 281–293 (2000)
Kirsch, A., Mitzenmacher, M.: Less hashing, same performance: Building a better Bloom filter. In: Azar, Y., Erlebach, T. (eds.) ESA 2006. LNCS, vol. 4168, pp. 456–467. Springer, Heidelberg (2006)
Manber, U., Wu, S.: An algorithm for approximate membership checking with application to password security. Information Processing Letters 50(4), 191–197 (1994)
Mitzenmacher, M.: Compressed Bloom filters. In: PODC 2001, pp. 144–150 (2001)
Moffat, A., Turpin, A.: Compression and Coding Algorithms. Kluwer Academic Publishers, Dordrecht (2002)
Pagh, A., Pagh, R., Rao, S.S.: An optimal Bloom filter replacement. In: SODA 2005, pp. 823–829 (2005)
Sanders, P., Transier, F.: Intersection in integer inverted indices. In: ALENEX 2007 (2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Putze, F., Sanders, P., Singler, J. (2007). Cache-, Hash- and Space-Efficient Bloom Filters. In: Demetrescu, C. (eds) Experimental Algorithms. WEA 2007. Lecture Notes in Computer Science, vol 4525. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72845-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-72845-0_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72844-3
Online ISBN: 978-3-540-72845-0
eBook Packages: Computer ScienceComputer Science (R0)