Skip to main content
SpringerLink
Log in

Flexible fingerprint cuckoo filter for information retrieval optimization in distributed network

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

In a large-scale distributed network, a naming service is used to achieve location transparency and provide effective content discovery. However, fast and accurate name retrieval in the massive name set is laborious. Approximate set membership data structures, such as Bloom filter and Cuckoo filter, are very popular in distributed information systems. They obtain high query performance and reduce memory requirements through the abstract representation of information, but at the cost of introducing query error rates, which will ultimately affect content service quality. In this paper, in order to obtain higher space utilization and a lower query false positive rate, we propose a flexible fingerprint cuckoo filter (FFCF) for information storage and retrieval, which can change the length and type of fingerprints adaptively. In our scheme, FFCF uses longer fingerprints under low occupancy and has the ability to correct errors by changing the type of stored fingerprints. Moreover, we give a theoretical proof and evaluate the performance of FFCF by experimental simulations with synthetic data sets and real network packets. The results demonstrate that FFCF can improve memory utilization, significantly reduce false positive errors by nearly 90\(\%\) at 50\(\%\) occupancy and outperform Cuckoo filter in the full range of occupancy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://github.com/Hawellian/FFCF.

  2. http://mawi.wide.ad.jp/mawi.

References

  1. Little, M.C., Shrivastava, S.K., Speirs, N.A.: Using bloom filters to speed-up name lookup in distributed systems. Comput. J. 45(6), 645–652 (2002). https://doi.org/10.1093/comjnl/45.6.645

    Article  Google Scholar 

  2. Koponen, T., Chawla, M., Chun, B., Ermolinskiy, A., Kim, K.H., Shenker, S., Stoica, I.: A data-oriented (and beyond) network architecture. In: Proceedings of the ACM SIGCOMM 2007 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 181–192. ACM, Kyoto (2007). https://doi.org/10.1145/1282380.1282402

  3. Zhang, L., Afanasyev, A., Burke, J., Jacobson, V., Claffy, K.C., Crowley, P., Papadopoulos, C., Wang, L., Zhang, B.: Named data networking. Comput. Commun. Rev. 44(3), 66–73 (2014). https://doi.org/10.1145/2656877.2656887

    Article  Google Scholar 

  4. Jinlin, W., Gang, C., Jiali, Y., Peng, S.: Seanet: architecture and technologies of an on-site, elastic, autonomous network. J. Netw. New Media 6, 1–8 (2020)

    Google Scholar 

  5. Venkataramani, A., Kurose, J.F., Raychaudhuri, D., Nagaraja, K., Mao, Z.M., Banerjee, S.: Mobilityfirst: a mobility-centric and trustworthy internet architecture. Comput. Commun. Rev. 44(3), 74–80 (2014). https://doi.org/10.1145/2656877.2656888

    Article  Google Scholar 

  6. Dharmapurikar, S., Krishnamurthy, P., Taylor, D.E.: Longest prefix matching using bloom filters. IEEE ACM Trans. Netw. 14(2), 397–409 (2006). https://doi.org/10.1145/1217619.1217632

    Article  Google Scholar 

  7. Abdulhassan, A.A., Ahmadi, M.: Many-field packet classification using CR-tree. J. High Speed Netw. 26(2), 125–140 (2020). https://doi.org/10.3233/JHS-200634

    Article  Google Scholar 

  8. Kwon, M., Reviriego, P., Pontarelli, S.: A Length-aware cuckoo filter for faster IP lookup. In: IEEE Conference on Computer Communications Workshops, INFOCOM, pp. 1071–1072. IEEE, San Francisco (2016). https://doi.org/10.1109/INFCOMW.2016.7562258

  9. Fan, L., Cao, P., Almeida, J.M., Broder, A.Z.: Summary cache: a scalable wide-area web cache sharing protocol. IEEE ACM Trans. Netw. 8(3), 281–293 (2000). https://doi.org/10.1109/90.851975

    Article  Google Scholar 

  10. Grashöfer, J., Jacob, F., Hartenstein, H.: Towards application of cuckoo filters in network security monitoring. In: 14th International Conference on Network and Service Management. CNSM, pp. 373–377. IEEE Computer Society, Rome (2018)

  11. Gao, W., Nguyen, J.H., Wu, Y., Hatcher, W.G., Yu, W.: Routing in large-scale dynamic networks: a bloom filter-based dual-layer scheme. ACM Trans. Internet Technol. 20(4), 38–13824 (2020). https://doi.org/10.1145/3407192

    Article  Google Scholar 

  12. Broder, A.Z., Mitzenmacher, M.: Survey: network applications of bloom filters: a survey. Internet Math. 1(4), 485–509 (2003). https://doi.org/10.1080/15427951.2004.10129096

    Article  MathSciNet  Google Scholar 

  13. Dai, H., Lu, J., Wang, Y., Pan, T., Liu, B.: BFAST: high-speed and memory-efficient approach for NDN forwarding engine. IEEE ACM Trans. Netw. 25(2), 1235–1248 (2017). https://doi.org/10.1109/TNET.2016.2623379

    Article  Google Scholar 

  14. Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970). https://doi.org/10.1145/362686.362692

    Article  Google Scholar 

  15. Bonomi, F., Mitzenmacher, M., Panigrahy, R., Singh, S., Varghese, G.: An improved construction for counting bloom filters. In: Algorithms—ESA 2006, 14th Annual European Symposium, vol. 4168, pp. 684–695. Springer, Zurich (2006). https://doi.org/10.1007/11841036_61

  16. Bender, M.A., Farach-Colton, M., Johnson, R., Kraner, R., Kuszmaul, B.C., Medjedovic, D., Montes, P., Shetty, P., Spillane, R.P., Zadok, E.: Don’t thrash: how to cache your hash on flash. Proc. VLDB Endow. 5(11), 1627–1637 (2012). https://doi.org/10.14778/2350229.2350275

    Article  Google Scholar 

  17. Rothenberg, C.E., Macapuna, C.A.B., Verdi, F.L., Magalhães, M.F.: The deletable bloom filter: a new member of the bloom family. IEEE Commun. Lett. 14(6), 557–559 (2010). https://doi.org/10.1109/LCOMM.2010.06.100344

    Article  Google Scholar 

  18. Fan, B., Andersen, D.G., Kaminsky, M., Mitzenmacher, M.: Cuckoo filter: practically better than bloom. In: Proceedings of the 10th ACM International on Conference on Emerging Networking Experiments and Technologies, pp. 75–88. ACM, Sydney (2014). https://doi.org/10.1145/2674005.2674994

  19. Pagh, R., Rodler, F.F.: Cuckoo hashing. J. Algorithms 51(2), 122–144 (2004). https://doi.org/10.1016/j.jalgor.2003.12.002

    Article  MathSciNet  Google Scholar 

  20. Wang, M., Zhou, M., Shi, S., Qian, C.: Vacuum filters: more space-efficient and faster replacement for bloom and cuckoo filters. Proc. VLDB Endow. 13(2), 197–210 (2019). https://doi.org/10.14778/3364324.3364333

    Article  Google Scholar 

  21. Reviriego, P., Martínez, J.A., Larrabeiti, D., Pontarelli, S.: Cuckoo filters and bloom filters: comparison and application to packet classification. IEEE Trans. Netw. Serv. Manag. 17(4), 2690–2701 (2020). https://doi.org/10.1109/TNSM.2020.3024680

    Article  Google Scholar 

  22. Mitzenmacher, M., Pontarelli, S., Reviriego, P.: Adaptive cuckoo filters. ACM J. Exp. Algorithmics 25, 1–20 (2020). https://doi.org/10.1145/3339504

    Article  MathSciNet  Google Scholar 

  23. Quan, W., Xu, C., Guan, J., Zhang, H., Grieco, L.A.: Scalable name lookup with adaptive prefix bloom filter for named data networking. IEEE Commun. Lett. 18(1), 102–105 (2014). https://doi.org/10.1109/LCOMM.2013.112413.132231

    Article  Google Scholar 

  24. Quan, W., Xu, C., Vasilakos, A.V., Guan, J., Zhang, H., Grieco, L.A.: TB2F: tree-bitmap and Bloom-filter for a scalable and efficient name lookup in content-centric networking. In: 2014 IFIP Networking Conference, pp. 1–9. IEEE Computer Society, Trondheim (2014). https://doi.org/10.1109/IFIPNetworking.2014.6857122

  25. Wang, Q., Wu, Q., Zhang, M., Zheng, R., Zhu, J.: Learned Bloom-filter for an efficient name lookup in information-centric networking. In: 2019 IEEE Wireless Communications and Networking Conference, pp. 1–6. IEEE, Marrakesh (2019). https://doi.org/10.1109/WCNC.2019.8885886

  26. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, vol. 27, pp. 3104–3112. MIT Press, Montreal (2014)

    Google Scholar 

  27. Erlingsson, U., Manasse, M., McSherry, F.: A cool and practical alternative to traditional hash tables. In: Proc. 7th Workshop on Distributed Data and Structures (WDAS’06), pp. 1–6. ACM, Santa Clara (2006)

  28. Eppstein, D.: Cuckoo filter: simplification and analysis. In: 15th Scandinavian Symposium and Workshops on Algorithm Theory, vol. 53, pp. 8–1812. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, Reykjavik (2016). https://doi.org/10.4230/LIPIcs.SWAT.2016.8

  29. Chen, H., Liao, L., Jin, H., Wu, J.: The dynamic cuckoo filter. In: 25th IEEE International Conference on Network Protocols, ICNP, pp. 1–10. IEEE Computer Society, Toronto (2017). https://doi.org/10.1109/ICNP.2017.8117563

  30. Breslow, A.D., Jayasena, N.: Morton filters: fast, compressed sparse cuckoo filters. VLDB J. 29(2–3), 731–754 (2020). https://doi.org/10.1007/s00778-019-00561-0

    Article  Google Scholar 

  31. Luo, L., Guo, D., Zhao, Y., Rottenstreich, O., Ma, R.T.B., Luo, X.: MCFsyn: a multi-party set reconciliation protocol with the marked cuckoo filter. IEEE Trans. Parallel Distrib. Syst. 32(11), 2705–2718 (2021). https://doi.org/10.1109/TPDS.2021.3074440

    Article  Google Scholar 

Download references

Acknowledgements

This work was funded by the Strategic Leadership Project of the Chinese Academy of Sciences: SEANET Technology Standardization Research System Development (Project No. XDC02070100) and the National Key R &D Program of China: Polymorphic Intelligent Network Environment (PINE) for Testing and Demonstrations (Project No. 2020YFB1806402).

Author information

Authors and Affiliations

  1. National Network New Media Engineering Research Center, Institute of Acoustics, Chinese Academy of Sciences, North Fourth Ring West Road, Beijing, 100190, China

    Wenhan Lian, Jinlin Wang & Jiali You

  2. University of Chinese Academy of Sciences, Yuquan Road, Beijing, 100049, China

    Wenhan Lian, Jinlin Wang & Jiali You

  3. Peng Cheng Lab, Xili Road, Shenzhen, 518055, China

    Jinlin Wang & Jiali You

Authors
  1. Wenhan Lian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinlin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiali You
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors listed contributed to the concept and design of the manuscript. WL and JY performed the experiment, wrote the manuscript and prepared the figures. JW performed the data analyses with constructive discussions and critically revised the manuscript.

Corresponding author

Correspondence to Jiali You.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lian, W., Wang, J. & You, J. Flexible fingerprint cuckoo filter for information retrieval optimization in distributed network. Distrib Parallel Databases (2024). https://doi.org/10.1007/s10619-024-07440-w

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10619-024-07440-w

Keywords

Search

Navigation