Stochastic analysis of a churn-tolerant structured peer-to-peer scheme



We present and analyze a simple and general scheme to build a churn (fault)-tolerant structured Peer-to-Peer (P2P) network. Our scheme shows how to “convert” a static network into a dynamic distributed hash table(DHT)-based P2P network such that all the good properties of the static network are guaranteed with high probability (w.h.p). Applying our scheme to a cube-connected cycles network, for example, yields a O(logN) degree connected network, in which every search succeeds in O(logN) hops w.h.p., using O(logN) messages, where N is the expected stable network size. Our scheme has an constant storage overhead (the number of nodes responsible for servicing a data item) and an O(logN) overhead (messages and time) per insertion and essentially no overhead for deletions. All these bounds are essentially optimal. While DHT schemes with similar guarantees are already known in the literature, this work is new in the following aspects: (1) It presents a rigorous mathematical analysis of the scheme under a general stochastic model of churn and shows the above guarantees; (2) The theoretical analysis is complemented by a simulation-based analysis that validates the asymptotic bounds even in moderately sized networks and also studies performance under changing stable network size; (3) The presented scheme seems especially suitable for maintaining dynamic structures under churn efficiently. In particular, we show that a spanning tree of low diameter can be efficiently maintained in constant time and logarithmic number of messages per insertion or deletion w.h.p.


P2P network DHT scheme Churn Dynamic spanning tree Stochastic analysis 


  1. 1.
    Abraham I, Awerbuch B, Azar Y, Bartal Y, Malkhi D, Pavlov E (2003) A generic scheme for building overlay networks in adversarial scenarios. In: IPDPSGoogle Scholar
  2. 2.
    Alon N, Spencer J (1992) The probabilistic method. WileyGoogle Scholar
  3. 3.
    Augustine J, Pandurangan G, Robinson P, Upfal E (2012) Towards robust and efficient computation in dynamic peer-to-peer networks. In: Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA)Google Scholar
  4. 4.
    Awerbuch B, Scheideler C (2006) Towards a scalable and robust DHT. In: Proceedings of ACM Symposium on Parallelism in Algorithms and Architectures (SPAA)Google Scholar
  5. 5.
    Balakrishnan H, Kaashoek F, Karger D, Morris R, Stoica, I (2003) Looking up data in P2P systems. Commun ACM 46(2):43–48CrossRefGoogle Scholar
  6. 6.
    Datar M (2002) Butterflies and p2p networks. In: Proceedings of the 10th European Symposium on Algorithms (ESA)Google Scholar
  7. 7.
    Elkin M (2007) A near-optimal fully dynamic distributed algorithm for maintaining sparse spanners. In: 26th ACM Symp. on Principles of Distributed Computing (PODC), pp 195–204Google Scholar
  8. 8.
    Falkner J, Piatek M, John JP, Krishnamurthy A, Anderson T (2007) Profiling a million user DHT. In: Proceedings of the 7th ACM SIGCOMM conference on internet measurementGoogle Scholar
  9. 9.
    Fiat A, Saia J (2007) Censorship Resistant Peer-to-Peer Content Addressable Networks. In: Proceedings of SODA, 2002. Journal version in Theory of Computing, vol 3, pp 1–23Google Scholar
  10. 10.
    Fris I, Havel I, Liebl P (1997) The diameter of the cube-connected cycles. Inf Process Lett 61(3):157–160MathSciNetCrossRefGoogle Scholar
  11. 11.
    Gavoille C, Peleg D (2003) Compact and localized distributed data structures. Distrib Comput 16(2-3):111–120CrossRefGoogle Scholar
  12. 12.
    Hildrum K, Kubiatowicz J (2003) Asymptotically efficient approaches to fault-tolerance in p2p networks. In: 17th International Symposium on Distributed Computing (DISC)Google Scholar
  13. 13.
    Jagannathan S, Pandurangan G, Srinvasan S (2006) Query protocols for highly resilient peer-to-peer networks. In: 19th international conference on parallel and distributed computing systemsGoogle Scholar
  14. 14.
    Kapron, BM, Kempe D, King V, Saia J, Sanwalani V (2010) Fast asynchronous byzantine agreement and leader election with full information. ACM TALG 6(4):68:1–68:28MathSciNetGoogle Scholar
  15. 15.
    Kashoek M, Karger D (2003) Koorde: a simple degree optimal distributed hash table. In: IPTPSGoogle Scholar
  16. 16.
    Kuhn F, Schmid S, Wattenhofer R (2005) Towards worst-case churn resistant peer-to-peer systems. Distrib Comput 22:249–267 (Conference version in IPTPS 2010)CrossRefGoogle Scholar
  17. 17.
    Leighton F (1992) Introduction to parallel algorithms and architectures. Morgan KaufmannGoogle Scholar
  18. 18.
    Liben-Nowell D, Balakrishnan H, Karger D (2002) Analysis of the evolution of peer-to-peer systems. In: Proceedings of ACM principles of distributed computingGoogle Scholar
  19. 19.
    Malkhi D, Naor M, Ratajczak D (2002) Viceroy: a scalable and dynamic emulation of the butterfly. In: ACM principles of distributed computingGoogle Scholar
  20. 20.
    Manku G (2003) Routing networks for distributed hash tables. In: Proceedings of the ACM principles of distributed computingGoogle Scholar
  21. 21.
    Maymounkov P, Mazieres D (2002) Kademlia. A peer-to-peer information system based on the XOR metric. In: Proc. of IPTPSGoogle Scholar
  22. 22.
    Mitzenmacher M, Upfal E (2005) Probability and computing. Cambridge University PressGoogle Scholar
  23. 23.
    Naor M, Weider U (2003) A simple faul-tolerant distributed hash table. In: Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS)Google Scholar
  24. 24.
    Naor M, Wieder U (2003) Novel architectures for p2p applications: the continuous-discrete approach. In: SPAA, pp 50–59Google Scholar
  25. 25.
    Pandurangan G, Raghavan P, Upfal E (2003) Building Low-Diameter P2P Networks. IEEE J Sel Areas Commun 21(6):995–1002 (Preliminary version in FOCS 2001)CrossRefGoogle Scholar
  26. 26.
    Peleg D (2000) Distributed computing: a locality-sensitive approach. SIAMGoogle Scholar
  27. 27.
    Ratnasamy S, Francis P, Handley M, Karp R, Shenker S (2001) A scalable content addressable network. In: Proceedings of ACM SIGCOMM 2001Google Scholar
  28. 28.
    Ross S (1970) Applied probability models with optimization applications. Dover PressGoogle Scholar
  29. 29.
    Rowstron A, Druschel P (2001) Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems . In: Proc. of the IFIP/ACM intenrational conference on distributed systems platforms, pp 329–350Google Scholar
  30. 30.
    Saia J, Fiat A, Gribble S, Karlin A, Saroiu S (2002) Dynamically fault-tolerant content addressable networks. In: Proceedings of the 1st international workshop on peer-to-peer systemsGoogle Scholar
  31. 31.
    Saroiu S, Gummadi P, Gribble S (2002) A measurement study of peer-to-peer file sharing systems. In: Proceedings of Multimedia Computing and Networking 2002 (MMCN’02). San Jose, CA, USAGoogle Scholar
  32. 32.
    Scheideler C, Schmid S (2009) A distributed and oblivious heap. In: Proceedings of ICALP, LNCS 5556, pp. 571–582Google Scholar
  33. 33.
    Sen S, Wang J (2004) Analyzing peer-to-peer traffic across large networks. IEEE/ACM Trans Netw 12(2):219–232CrossRefGoogle Scholar
  34. 34.
    Shen H, Xu C-Z, Chen G (2006) Cycloid: a constant-degree and lookup-efficient p2p overlay network. Perform Eval 63(3):195–216CrossRefGoogle Scholar
  35. 35.
    Stoica I, Morris R, Karger D, Kaashoek F, Balakrishnan H (2001) Chord: a scalable peer-to-peer lookup service for internet applications. In: Proceedings of the 2001 ACM SIGCOMM conference, pp 149–160Google Scholar
  36. 36.
    Stutzbach D, Rejaie R (2006) Understanding churn in peer-to-peer networks. In: Proceedings of the 6th ACM SIGCOMM conference on internet measurementGoogle Scholar
  37. 37.
    Stutzbach D, Rejaie R, Duffield N, Sen S, Willinger W (2006) On unbiased sampling for unstructured peer-to-peer networks. In: Proceedings of the 6th ACM SIGCOMM conference on internet measurementGoogle Scholar
  38. 38.
    Zhao B, Kubiatowicz J, Joseph A (2001) Tapestry: an infrastructure for fault-tolerant wide-area location and routing. Technical Report UCB/CSD-01-1141, UC BerkeleyGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2012

Authors and Affiliations

  1. 1.Department of Computer SciencePurdue UniversityWest LafayetteUSA
  2. 2.Division of Mathematical SciencesNanyang Technological UniversitySingaporeSingapore
  3. 3.Department of Computer ScienceBrown UniversityProvidenceUSA

Personalised recommendations