Advertisement

The Gaussian Bloom Filter

  • Martin WernerEmail author
  • Mirco Schönfeld
Conference paper
  • 1.5k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9049)

Abstract

Modern databases tailored to highly distributed, fault tolerant management of information for big data applications exploit a classical data structure for reducing disk and network I/O as well as for managing data distribution: The Bloom filter. This data structure allows to encode small sets of elements, typically the keys in a key-value store, into a small, constant-size data structure. In order to reduce memory consumption, this data structure suffers from false positives which lead to additional I/O operations and are therefore only harmful with respect to performance. With this paper, we propose an extension to the classical Bloom filter construction which facilitates the use of floating point coprocessors and GPUs or additional main memory in order to reduce false positives. The proposed data structure is compatible with the classical construction in the sense that the classical Bloom filter can be extracted in time linear to the size of the data structure and that the Bloom filter is a special case of our construction. We show that the approach provides a relevant gain with respect to the false positive rate. Implementations for Apache Cassandra, C++, and NVIDIA CUDA are given and support the feasibility and results of the approach.

Keywords

Bloom Filter Database design Data structures 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    The Apache Cassandra database (2014). http://cassandra.apache.org/
  2. 2.
    Appleby, A.: Murmurhash (2009). http://code.google.com/p/smhasher/
  3. 3.
    Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Communications of the ACM 13(7), 422–426 (1970)CrossRefzbMATHGoogle Scholar
  4. 4.
    Broder, A., Mitzenmacher, M.: Network applications of bloom filters: A survey. Internet Mathematics 1(4), 485–509 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Byers, J., Considine, J., Mitzenmacher, M.: Fast approximate reconciliation of set differences. Tech. rep., Boston University Computer Science Department (2002)Google Scholar
  6. 6.
    Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS) 26(2), 4 (2008)CrossRefGoogle Scholar
  7. 7.
    Eastlake, D., Jones, P.: US Secure Hash Algorithm 1 (SHA1) (2001)Google Scholar
  8. 8.
    Feng, W.C., Kandlur, D.D., Saha, D., Shin, K.G.: Stochastic fair blue: a queue management algorithm for enforcing fairness. In: Proceedings of the Twentieth Annual Joint Conference of the IEEE Computer and Communications Societies, INFOCOM 2001, vol. 3, pp. 1520–1529. IEEE (2001)Google Scholar
  9. 9.
    Nakamoto, S.: Bitcoin: A peer-to-peer electronic cash system (2008)Google Scholar
  10. 10.
    Schönfeld, M., Werner, M.: Node wake-up via ovsf-coded bloom filters in wireless sensor networks. In: Ad Hoc Networks, pp. 119–134. Springer (2014)Google Scholar
  11. 11.
    Schraudolph, N.N.: A fast, compact approximation of the exponential function. Neural Computation 11(4), 853–862 (1999)CrossRefGoogle Scholar
  12. 12.
    Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 26th IEEE Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–10 (2010)Google Scholar
  13. 13.
    Whitaker, A., Wetherall, D.: Forwarding without loops in icarus. In: 2002 IEEE Open Architectures and Network Programming Proceedings, pp. 63–75. IEEE (2002)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Ludwig-Maximilians-Universität in MunichMunichGermany

Personalised recommendations