Abstract
We consider the problem of managing a dynamic heterogeneous storage system in a distributed way so that the amount of data assigned to a host in that system is related to its capacity. Two central problems have to be solved for this: (1) organizing the hosts in an overlay network with low degree and diameter so that one can efficiently check the correct distribution of the data and route between any two hosts, and (2) distributing the data among the hosts so that the distribution respects the capacities of the hosts and can easily be adapted as the set of hosts or their capacities change. We present distributed protocols for these problems that are self-stabilizing and that do not need any global knowledge about the system such as the number of nodes or the overall capacity of the system. Prior to this work no solution was known satisfying these properties.
This work was partially supported by the German Research Foundation (DFG) within the Collaborative Research Centre ”On-The-Fly Computing” (SFB 901).
The full version of the paper can be found in [32].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dijkstra, E.W.: Self-stabilizing systems in spite of distributed control. Commun. ACM 17, 643–644 (1974)
Schindelhauer, C., Schomaker, G.: Weighted distributed hash tables. In: SPAA 2005, pp. 218–227 (2005)
Giakkoupis, G., Hadzilacos, V.: A Scheme for Load Balancing in Heterogenous Distributed Hash Tables. In: PODC 2005 (2005)
Brinkmann, A., Salzwedel, K., Scheideler, C.: Compact, adaptive placement schemes for non-uniform distribution requirements. In: SPAA 2002, pp. 53–62 (2002)
Brinkmann, A., Salzwedel, K., Scheideler, C.: Efficient, distributed data placement strategies for storage area networks. In: SPAA 2000, pp. 119–128 (2000)
Karger, D., Lehman, E., Leighton, T., Levine, M., Lewin, D., Panigrahy, R.: Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web. In: STOC 1997, pp. 654–663 (1997)
Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable content-addressable network. In: SIGCOMM, pp. 161–172 (2001)
Rowstron, A., Druschel, P.: Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In: Guerraoui, R. (ed.) Middleware 2001. LNCS, vol. 2218, pp. 329–350. Springer, Heidelberg (2001)
Stoica, I., Morris, R., Karger, D., Frans Kaashoek, M., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for internet applications. In: SIGCOMM, pp. 149–160 (2001)
Cramer, C., Fuhrmann, T.: Self-stabilizing ring networks on connected graphs. In: Technical report, University of Karlsruhe (TH), Fakultaet fuer Informatik (May 2005)
Dolev, S., Tzachar, N.: Empire of colonies: Self-stabilizing and self-organizing distributed algorithm. Theor. Comput. Sci. 410(6-7), 514–532 (2009)
Jacob, R., Richa, A.W., Scheideler, C., Schmid, S., Täubig, H.: A distributed polylogarithmic time algorithm for self-stabilizing skip graphs. In: PODC 2009, pp. 131–140 (2009)
Jacob, R., Ritscher, S., Scheideler, C., Schmid, S.: A self-stabilizing and local delaunay graph construction. In: Dong, Y., Du, D.-Z., Ibarra, O. (eds.) ISAAC 2009. LNCS, vol. 5878, pp. 771–780. Springer, Heidelberg (2009)
Kniesburges, S., Koutsopoulos, A., Scheideler, C.: Re-chord: a self-stabilizing chord overlay network. In: SPAA 2011, pp. 235–244 (2011)
Nor, R.M., Nesterenko, M., Scheideler, C.: Corona: A stabilizing deterministic message-passing skip list. In: Défago, X., Petit, F., Villain, V. (eds.) SSS 2011. LNCS, vol. 6976, pp. 356–370. Springer, Heidelberg (2011)
Onus, M., Richa, A.W., Scheideler, C.: Linearization: Locally self-stabilizing sorting in graphs. In: ALENEX 2007, pp. 99–108 (2007)
Harvey, N.: CPSC 536N: Randomized Algorithms, Pages 5. Lecture 3. University of British Columbia (2011-2012)
Shena, H., Xub, C.-Z.: Hash-based proximity clustering for efficient load balancing in heterogeneous DHT networks. J. Parallel Distrib. Comput. 68, 686–702 (2008)
Rao, A., Lakshminarayanan, K., Surana, S., Karp, R., Stoica, I.: Load balancing in structured P2P systems. In: Kaashoek, M.F., Stoica, I. (eds.) IPTPS 2003. LNCS, vol. 2735, Springer, Heidelberg (2003)
Godfrey, P.B., Stoica, I.: Heterogeneity and Load Balance in Distributed Hash Tables. In: IEEE INFOCOM (2005)
Bienkowski, M., Brinkmann, A., Klonowski, M., Korzeniowski, M.: SkewCCC+: A heterogeneous distributed hash table. In: Lu, C., Masuzawa, T., Mosbah, M. (eds.) OPODIS 2010. LNCS, vol. 6490, pp. 219–234. Springer, Heidelberg (2010)
Santos, J.R., Muntz, R.: Performance Analysis of the RIO Multimedia Storage System with Heterogeneous Disk Configurations. In: ACM Multimedia Conference 1998, pp. 303–308 (1998)
Miranda, A., Effert, S., Kang, Y., Miller, E.L., Brinkmann, A., Cortes, T.: Reliable and randomized data distribution strategies for large scale storage systems. In: HiPC 2011, pp. 1–10 (2011)
Didi Yao, S.-Y., Shahabi, C., Zimmermann, R.: BroadScale: Efficient scaling of heterogeneous storage systems. Int. J. on Digital Libraries 6, 98–111 (2006)
Brinkmann, A., Effert, S., Meyer auf der Heide, F., Scheideler, C.: Dynamic and Redundant Data Placement. In: ICDCS 2007, p. 29 (2007)
Cortes, T., Labarta, J.: Taking advantage of heterogeneity in disk arrays. J. Parallel Distrib. Comput. 63, 448–464 (2003)
Mense, M., Scheideler, C.: SPREAD: An adaptive scheme for redundant and fair storage in dynamic heterogeneous storage systems. In: SODA 2008 (2008)
Awerbuch, B., Scheideler, C.: The hyperring: a low-congestion deterministic data structure for distributed environments. In: SODA 2004, pp. 318–327 (2004)
Bhargava, A., Kothapalli, K., Riley, C., Scheideler, C., Thober, M.: Pagoda: A dynamic overlay network for routing, data management, and multicasting. In: SPAA 2004, pp. 170–179 (2004)
Kuhn, F., Schmid, S., Wattenhofer, R.: A self-repairing peer-to-peer system resilient to dynamic adversarial churn. In: van Renesse, R. (ed.) IPTPS 2005. LNCS, vol. 3640, pp. 13–23. Springer, Heidelberg (2005)
Plaxton, C.G., Rajaraman, R., Richa, A.W.: Accessing nearby copies of replicated objects in a distributed environment. In: SPAA 1997, pp. 311–320 (1997)
Kniesburges, S., Koutsopoulos, A., Scheideler, C.: CONE-DHT: A distributed self-stabilizing algorithm for a heterogeneous storage system (Pre-Print) In: arXiv:1307.6747 [cs.DC] (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kniesburges, S., Koutsopoulos, A., Scheideler, C. (2013). CONE-DHT: A Distributed Self-Stabilizing Algorithm for a Heterogeneous Storage System. In: Afek, Y. (eds) Distributed Computing. DISC 2013. Lecture Notes in Computer Science, vol 8205. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41527-2_37
Download citation
DOI: https://doi.org/10.1007/978-3-642-41527-2_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41526-5
Online ISBN: 978-3-642-41527-2
eBook Packages: Computer ScienceComputer Science (R0)