Simple Load Balancing for Distributed Hash Tables

  • John Byers
  • Jeffrey Considine
  • Michael Mitzenmacher
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2735)

Abstract

Distributed hash tables have recently become a useful building block for a variety of distributed applications. However, current schemes based upon consistent hashing require both considerable implementation complexity and substantial storage overhead to achieve desired load balancing goals. We argue in this paper that these goals can be achieved more simply and more cost-effectively. First, we suggest the direct application of the “power of two choices” paradigm, whereby an item is stored at the less loaded of two (or more) random alternatives. We then consider how associating a small constant number of hash values with a key can naturally be extended to support other load balancing strategies, including load-stealing or load-shedding, as well as providing natural fault-tolerance mechanisms.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Azar, Y., Broder, A., Karlin, A., Upfal, E.: Balanced allocations. SIAM Journal on Computing 29(1), 180–200 (1999)CrossRefMathSciNetMATHGoogle Scholar
  2. 2.
    Byers, J., Considine, J., Mitzenmacher, M., Rost, S.: Informed content delivery across adaptive overlay networks. In: SIGCOMM, pp. 47–60 (2002)Google Scholar
  3. 3.
    Byers, J.W., Luby, M., Mitzenmacher, M.: Accessing multiple mirror sites in parallel: Using tornado codes to speed up downloads. In: INFOCOM, vol. (1), pp. 275–283 (1999)Google Scholar
  4. 4.
    Clark, D.: The design philosophy of the DARPA internet protocols. In: ACM SIGCOMM, pp. 106–114. ACM Press, New York (1988)Google Scholar
  5. 5.
    Dabek, F., Kaashoek, M.F., Karger, D., Morris, R., Stoica, I.: Widearea cooperative storage with CFS. In: Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP 2001), Chateau Lake Louise, Banff, Canada (October 2001)Google Scholar
  6. 6.
    Karger, D.R., Lehman, E., Leighton, F.T., Panigrahy, R., Levine, M.S., Lewin, D.: Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the world wide web. In: ACM Symposium on Theory of Computing, May 1997, pp. 654–663 (1997)Google Scholar
  7. 7.
    Mitzenmacher, M., Richa, A., Sitaraman, R.: The Power of Two Choices: A Survey of Techniques and Results. Pardalos, P., Rajasekaran, S., Reif, J., Rolim, J. (eds.), pp. 255–312. Kluwer Academic Publishers, Norwell (2001)Google Scholar
  8. 8.
    Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable content addressable network. In: ACM SIGCOMM, pp. 161–172 (2001)Google Scholar
  9. 9.
    Rowstron, A., Druschel, P.: Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In: Guerraoui, R. (ed.) Middleware 2001. LNCS, vol. 2218, p. 329. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  10. 10.
    Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for internet applications. In: ACM SIGCOMM, pp. 149–160 (2001)Google Scholar
  11. 11.
    Vöcking, B.: How asymmetry helps load balancing. In: Proceedings of the 40th IEEE-FOCS, pp. 131–140 (1999)Google Scholar
  12. 12.
    Zhao, B.Y., Kubiatowicz, J.D., Joseph, A.D.: Tapestry: An infrastructure for fault-tolerant wide-area location and routing. Tech. Rep. UCB/CSD-01-1141, UC Berkeley (April 2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • John Byers
    • 1
  • Jeffrey Considine
    • 1
  • Michael Mitzenmacher
    • 2
  1. 1.Department of Computer ScienceBoston UniversityBostonUSA
  2. 2.EECSHarvard UniversityCambridgeUSA

Personalised recommendations