High Availability in DHTs: Erasure Coding vs. Replication

  • Rodrigo Rodrigues
  • Barbara Liskov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3640)

Abstract

High availability in peer-to-peer DHTs requires data redundancy. This paper compares two popular redundancy schemes: replication and erasure coding. Unlike previous comparisons, we take the characteristics of the nodes that comprise the overlay into account, and conclude that in some cases the benefits from coding are limited, and may not be worth its disadvantages.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., Zhao, B.: OceanStore: An architecture for global-scale persistent storage. In: ASPLOS-IX: Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, pp. 190–201 (2000)Google Scholar
  2. 2.
    Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable content-addressable network. In: SIGCOMM 2001: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, San Diego, California, United States, pp. 161–172 (2001)Google Scholar
  3. 3.
    Rowstron, A., Druschel, P.: Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. In: Proceedings of the 18th ACM Symposium on Operating System Principles, Banff, Canada (2001)Google Scholar
  4. 4.
    Dabek, F., Li, J., Sit, E., Robertson, J., Kaashoek, F., Morris, R.: Designing a DHT for low latency and high throughput. In: Proceedings of the First ACM/Usenix Symposium on Networked Systems Design and Implementation (NSDI), San Francisco, California (2004)Google Scholar
  5. 5.
    Bhagwan, R., Tati, K., Cheng, Y.C., Savage, S., Voelker, G.: In: Proceedings of the First ACM/Usenix Symposium on Networked Systems Design and Implementation (NSDI), San Francisco, California, United States (2004)Google Scholar
  6. 6.
    Weatherspoon, H., Kubiatowicz, J.D.: Erasure coding vs. replication: A quantitative comparison. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, p. 328. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Blake, C., Rodrigues, R.: High availability, scalable storage, dynamic peer networks: Pick two. In: Proceedings of The Ninth Workshop on Hot Topics in Operating Systems (HotOS-IX), Lihue, Hawaii, United States (2003)Google Scholar
  8. 8.
    Bhagwan, R., Savage, S., Voelker, G.: Understanding availability. In: Kaashoek, M.F., Stoica, I. (eds.) IPTPS 2003. LNCS, vol. 2735, Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Reed, S., Solomon, G.: Polynomial codes over certain finite fields. J. SIAM 8, 300–304 (1960)Google Scholar
  10. 10.
    Rabin, M.: Efficient dispersal of information for security, load balancing, and fault tolerance. J. ACM 36 (1989)Google Scholar
  11. 11.
    Luby, M.: LT codes. In: Proceedings of the 43rd Symposium on Foundations of Computer Science (FOCS 2002), Vancouver, Canada (2002)Google Scholar
  12. 12.
    Andersen, D.: Improving End-to-End Availability Using Overlay Networks. PhD thesis, MIT (2005)Google Scholar
  13. 13.
    Karger, D., Lehman, E., Leighton, T., Levine, M., Lewin, D., Panigrahy, R.: Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web. In: Proc. 29th Symposium on Theory of Computing, El Paso, Texas, pp. 654–663 (1997)Google Scholar
  14. 14.
    Bertsekas, D., Gallager, R.: Data Networks. Prentice-Hall, Englewood Cliffs (1987)Google Scholar
  15. 15.
    Rhea, S., Geels, D., Roscoe, T., Kubiatowicz, J.: Handling churn in a DHT. In: Proceedings of the USENIX Annual Technical Conference (2004)Google Scholar
  16. 16.
    Bolosky, W.J., Douceur, J.R., Ely, D., Theimer, M.: Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs. In: Proceedings of the international conference on measurement and modeling of computer systems (SIGMETRICS), pp. 34–43 (2000)Google Scholar
  17. 17.
    Stribling, J.: Planetlab - all pairs pings (2005), http://pdos.lcs.mit.edu/~strib/pl_app

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Rodrigo Rodrigues
    • 1
  • Barbara Liskov
    • 2
  1. 1.INESC-ID / Instituto Superior TécnicoLisbonPortugal
  2. 2.MIT Computer Science and Artificial Intelligence LaboratoryCambridgeUSA

Personalised recommendations