Skip to main content
SpringerLink
Log in

Improving lookup reliability in Kad

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

Kad is one of the most popular peer-to-peer (P2P) networks deployed on today’s Internet. It provides support for file-sharing applications such as eMule and aMule and serves millions of users. Its reliability impacts not only the availability of file-sharing services, but also the capability of supporting other Internet services. However, in today’s Kad network, its lookup operation’s success ratio is lower than 91 % and not suitable for critical applications. In this paper, we investigate why Kad lookup fails and propose several new solutions. We build a measurement system called Anthill to analyze Kad’s communication process quantitatively, and figure out that the causes of Kad’s lookup failures can be classified into four categories: packet loss, selective Denial of Service nodes, search sequence miss, and publish/search space miss. The first two are due to the environment changes, the third is caused by the detachment of routing operations and content operations in Kad, and the last one shows the limitations of the Kademlia DHT algorithm under Kad’s current configuration. Based on the analysis, we propose corresponding approaches for Kad, including packet-retransmission, neighborhood lookup, and β-adjusting. We have systematically measured the effectiveness and efficiency of these approaches, and then give several recommendations for adoption in different situations. The improved version of Kad can achieve a success ratio of 99.8 % for lookup operations, with only a moderate communication overhead, while its average lookup latency is reduced significantly to only about 1 second. Our work shows that, with proper configurations and improvements, Kad can work much better and is capable of supporting more Internet services.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. aMule web site (2011) http://www.amule.org/

  2. eMule web site (2011) http://www.emule-project.net

  3. An industry-leading provider of ip intelligence and online fraud detection tools (2012) http://www.maxmind.com

  4. Bryan DA, Lowekamp BB, Jennings C (2005) Sosimple: a serverless, standards-based, p2p sip communication system. In: Proceedings of the 1st international workshop on advanced architectures and algorithms for internet delivery and applications (AAA-IDEA). IEEE Computer Society, Washington, pp 42–49

    Google Scholar 

  5. Carra D, Biersack EW (2007) Building a reliable p2p system out of unreliable p2p clients: the case of kad. In: Proceedings of the 2007 ACM CoNEXT conference. ACM, New York, pp 28:1–28:12

    Google Scholar 

  6. Cholez T, Henard C, Chrisment I, Festor O, Doyen G, Khatoun R (2011) A first approach to detect suspicious peers in the kad p2p network. In: 2011 Conference on network and information systems security (SAR-SSI), pp 1–8

  7. Cohen B (2003) Incentives build robustness in bittorrent. In: First workshop on economics of peer-to-peer systems. Berkeley

  8. Dabek F, Li J, Sit E, Robertson J, Kaashoek MF, Morris R (2004) Designing a dht for low latency and high throughput. In: Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation (NSDI). USENIX Association, Berkeley, pp 1–14

    Google Scholar 

  9. Freedman MJ, Freudenthal E, Mazières D (2004) Democratizing content publication with coral. In: Proceedings of the 1st conference on symposium on networked systems design and implementation (NSDI). USENIX Association, Berkeley, pp 18–18

    Google Scholar 

  10. Kang H, Chan-Tin E, Hopper N, Kim Y (2009) Why kad lookup fails. In: Proceedings of IEEE 9th international conference on peer-to-peer computing (P2P). IEEE, pp 121–130

  11. Li J, Stribling J, Morris R, Kaashoek M, Gil T (2005) A performance vs. cost framework for evaluating dht design tradeoffs under churn. In: Proceedings of IEEE 24th annual joint conference of the ieee computer and communications societies (INFOCOM), vol 1, pp 225–236

  12. Liu B, Wei T, Zhang J, Li J, Zou W, Zhou M (2012) Revisiting why kad lookup fails. In: Proceedings of IEEE 12th international conference on peer-to-peer computing (P2P). IEEE, pp 37–42

  13. Lopez PG, Artigas MS, Ahull JP (2007) The p2pweb model: A glue for the web. In: Proceedings of the 16th IEEE international workshops on enabling technologies: infrastructure for collaborative enterprises (WETICE). IEEE Computer Society, Washington, pp 153–158

    Google Scholar 

  14. Maymounkov P, Mazières D (2002) Kademlia: a peer-to-peer information system based on the xor metric. In: Proceedings of the 1st International Workshop on Peer-to-Peer Systems (IPTPS). Springer-Verlag, London, pp 53–65

    Chapter  Google Scholar 

  15. Mol JJD, Pouwelse JA, Epema DHJ, Sips HJ (2008) Free-riding, fairness, and firewalls in p2p file-sharing. In: Proceedings of the 8th International conference on peer-to-peer computing (P2P). IEEE Computer Society, Washington, pp 301–310

    Google Scholar 

  16. Nakamoto S (2008) Bitcoin: A peer-to-peer electronic cash system

  17. Ramasubramanian V, Sirer EG (2004) Beehive: O(1)lookup performance for power-law query distributions in peer-to-peer overlays. In: Proceedings of the 1st conference on symposium on networked systems design and implementation (NSDI). USENIX Association, Berkeley, pp 1–14

    Google Scholar 

  18. Ramasubramanian V, Sirer EG (2004) The design and implementation of a next generation name service for the internet. SIGCOMM Comput Commun Rev 34(4):331–342

    Article  Google Scholar 

  19. Ratnasamy S, Stoica I, Shenker S (2002) Routing algorithms for dhts: some open questions. In: Proceedings of the 1st international workshop on peer-to-peer systems (IPTPS), IPTPS ’01. Springer-Verlag, London, pp 45–52

    Google Scholar 

  20. Rhea S, Geels D, Roscoe T, Kubiatowicz J (2004) Handling churn in a dht. In: Proceedings of the annual conference on USENIX annual technical conference (ATEC), ATEC ’04. USENIX Association, Berkeley, pp 1–14

    Google Scholar 

  21. Risson J, Moors T (2006) Survey of research towards robust peer-to-peer networks: search methods. Comput Netw 50(17):3485–3521

    Article  MATH  Google Scholar 

  22. Steiner M, Biersack EW, Ennajjary T (2007) Actively monitoring peers in kad. In: Proceedings of the 6th international workshop on peer-to-peer systems (IPTPS). Bellevue

  23. Steiner M, Carra D, Biersack EW (2008) Faster content access in kad. In: Proceedings of the 8th international conference on peer-to-peer computing (P2P), P2P ’08. IEEE Computer Society, Washington, pp 195–204

    Google Scholar 

  24. Steiner M, En-Najjary T, Biersack EW (2007) A global view of kad. In: Proceedings of the 7th ACM SIGCOMM conference on internet measurement (IMC). ACM, New York, pp 117–122

    Google Scholar 

  25. Steiner M, En-Najjary T, Biersack EW (2009) Long term study of peer behavior in the kad dht. IEEE/ACM Trans Netw 17(5):1371–1384

    Article  Google Scholar 

  26. Stutzbach D, Rejaie R (2006) Improving lookup performance over a widely-deployed dht. In: Proceedings of the 25th IEEE international conference on computer communications (INFOCOM), pp. 1–12

  27. Stutzbach D, Rejaie R (2006) Understanding churn in peer-to-peer networks. In: Proceedings of the 6th ACM SIGCOMM conference on Internet measurement (IMC). ACM, New York, pp 189–202

    Google Scholar 

  28. The ddos attack on revision3 (2008) http://revision3.com/blog/2008/05/29/inside-the-attack-that-crippled-revision3/

  29. The famed anti-p2p company (2012) http://en.wikipedia.org/wiki/MediaDefender

  30. Yu J, Li Z (2009) Active measurement of routing table in kad. In: Proceedings of the 6th IEEE conference on consumer communications and networking conference (CCNC). IEEE Press, Piscataway, pp 1252–1256

    Google Scholar 

Download references

Acknowledgments

This research was supported in part by National Natural Science Foundation of China (Grant No. 61003216) and Research project of mal-behavior and privacy leaking detecting technologies on mobile Internet(Grant No. Z111101055311052).

Author information

Authors and Affiliations

  1. Peking University, Beijing, 100871, China

    Bingshuang Liu, Tao Wei, Chao Zhang & Jianyu Zhang

  2. University of California, Berkeley, CA, 94720, USA

    Tao Wei

  3. University of Oregon, Eugene, OR, 97403, USA

    Jun Li

Authors
  1. Bingshuang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Wei.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, B., Wei, T., Zhang, C. et al. Improving lookup reliability in Kad. Peer-to-Peer Netw. Appl. 8, 156–170 (2015). https://doi.org/10.1007/s12083-013-0240-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-013-0240-4

Keywords

Search

Navigation