Retrieval and Perfect Hashing Using Fingerprinting

  • Ingo Müller
  • Peter Sanders
  • Robert Schulze
  • Wei Zhou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8504)

Abstract

Recent work has shown that perfect hashing and retrieval of data values associated with a key can be done in such a way that there is no need to store the keys and that only a few bits of additional space per element are needed. We present FiRe – a new, very simple approach to such data structures. FiRe allows very fast construction and better cache efficiency. The main idea is to substitute keys by small fingerprints. Collisions between fingerprints are resolved by recursively handling those elements in an overflow data structure. FiRe is dynamizable, easily parallelizable and allows distributed implementation without communicating keys. Depending on implementation choices, queries may require close to a single access to a cache line or the data structure needs as low as 2.58 bits of additional space per element.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Belazzougui, D., Botelho, F.C., Dietzfelbinger, M.: Hash, displace, and compress. In: Fiat, A., Sanders, P. (eds.) ESA 2009. LNCS, vol. 5757, pp. 682–693. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Botelho, F.C., Pagh, R., Ziviani, N.: Simple and space-efficient minimal perfect hash functions. In: Dehne, F., Sack, J.-R., Zeh, N. (eds.) WADS 2007. LNCS, vol. 4619, pp. 139–150. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Botelho, F.C., Ziviani, N.: External perfect hashing for very large key sets. In: 16th ACM Conference on Information and Knowledge Management, pp. 653–662. ACM, New York (2007)Google Scholar
  4. 4.
    de Castro Reis, D., Belazzougui, D., Botelho, F.C., Ziviani, N.: CMPH – C Minimal Perfect Hashing Library, http://cmph.sf.net
  5. 5.
    Demaine, E.D., der Heide, F.M.A., Pagh, R., Pǎtraşcu, M.: De dictionariis dynamicis pauco spatio utentibus. In: Correa, J.R., Hevia, A., Kiwi, M. (eds.) LATIN 2006. LNCS, vol. 3887, pp. 349–361. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Dietzfelbinger, M., Weidling, C.: Balanced allocation and dictionaries with tightly packed constant size bins. Theoret. Comput. Sci. 380(1-2), 47–68 (2007), http://dx.doi.org/10.1016/j.tcs.2007.02.054 CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Dietzfelbinger, M.: Design strategies for minimal perfect hash functions. In: Hromkovič, J., Královič, R., Nunkesser, M., Widmayer, P. (eds.) SAGA 2007. LNCS, vol. 4665, pp. 2–17. Springer, Heidelberg (2007), http://dx.doi.org/10.1007/978-3-540-74871-7_2 CrossRefGoogle Scholar
  8. 8.
    Dietzfelbinger, M., Pagh, R.: Succinct data structures for retrieval and approximate membership (Extended abstract). In: Aceto, L., Damgård, I., Goldberg, L.A., Halldórsson, M.M., Ingólfsdóttir, A., Walukiewicz, I. (eds.) ICALP 2008, Part I. LNCS, vol. 5125, pp. 385–396. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Edelkamp, S., Sanders, P., Šimeček, P.: Semi-external LTL model checking. In: Gupta, A., Malik, S. (eds.) CAV 2008. LNCS, vol. 5123, pp. 530–542. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Eppstein, D., Goodrich, M.: Straggler identification in round-trip data streams via newton’s identities and invertible Bloom filters. IEEE Trans. Knowl. Data Eng. 23(2), 297–306 (2011)CrossRefGoogle Scholar
  11. 11.
    Fan, B., Andersen, D.G., Kaminsky, M.: Cuckoo filter: Better than bloom. Login 38(4) (2013)Google Scholar
  12. 12.
    Färber, F., et al.: SAP HANA Database: Data management for modern business applications. SIGMOD Rec. 40(4), 45–51 (2012), http://doi.acm.org/10.1145/2094114.2094126 CrossRefGoogle Scholar
  13. 13.
    Fredriksson, K., Nikitin, F.: Simple compression code supporting random access and fast string matching. In: Demetrescu, C. (ed.) WEA 2007. LNCS, vol. 4525, pp. 203–216. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
  15. 15.
    Hagerup, T., Tholey, T.: Efficient minimal perfect hashing in nearly minimal space. In: Ferreira, A., Reichel, H. (eds.) STACS 2001. LNCS, vol. 2010, pp. 317–326. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  16. 16.
    Jenkins, B.: Algorithm alley: Hash functions. Dr. Dobb’s Journal (1997)Google Scholar
  17. 17.
    Karp, R.M., Rabin, M.O.: Efficient randomized pattern-matching algorithms. IBM J. Res. Dev. 31(2), 249–260 (1987)CrossRefMATHMathSciNetGoogle Scholar
  18. 18.
    Lim, H., Andersen, D.G., Kaminsky, M.: Practical batch-updatable external hashing with sorting. In: ALENEX, pp. 173–182. SIAM, Philadelphia (2013)Google Scholar
  19. 19.
    Navarro, G., Providel, E.: Fast, small, simple rank/select on bitmaps. In: Klasing, R. (ed.) SEA 2012. LNCS, vol. 7276, pp. 295–306. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  20. 20.
    Wassenberg, J., Sanders, P.: Engineering a multi-core radix sort. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011, Part II. LNCS, vol. 6853, pp. 160–169. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  21. 21.
    Zhou, W.: A Compact Cache-Efficient Function Store with Constant Evaluation Time. Bachelor thesis, KIT and SAP (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Ingo Müller
    • 1
    • 2
  • Peter Sanders
    • 1
  • Robert Schulze
    • 2
  • Wei Zhou
    • 1
    • 2
  1. 1.Karlsruhe Institute of TechnologyKarlsruheGermany
  2. 2.SAP AGWalldorfGermany

Personalised recommendations