RDIM: A Self-adaptive and Balanced Distribution for Replicated Data in Scalable Storage Clusters
- 530 Downloads
As storage systems scale from a few storage nodes to hundreds or thousands, data distribution and load balancing become increasingly important. We present a novel decentralized algorithm, RDIM (Replication Under Dynamic Interval Mapping), which maps replicated objects to a scalable collection of storage nodes. RDIM distributes objects to nodes evenly, redistributing as few objects as possible when new nodes are added or existing nodes are removed to preserve this balanced distribution. It supports weighted allocation and guarantees that replicas of a particular object are not placed on the same node. Its time complexity and storage requirements compare favorably with known methods.
KeywordsData Object Storage Node Mapping Storage Data Replication Balance Distribution
Unable to display preview. Download preview PDF.
- 1.Xin, Q., Miller, E.L., Long, D.D.E., Brandt, S.A., Schwarz, T., Litwin, W.: Reliability mechanisms for very large storage systems. In: Proceedings of the 20th IEEE / 11th NASA Goddard Conference on Mass Storage Systems and Technologies, April 2003, pp. 146–156 (2003)Google Scholar
- 3.Devine, R.: Design and implementation of DDH: A distributed dynamic hashing algorithm. In: Proceedings of the 4th International Conference on Foundations of Data Organization and Algorithms, pp. 101–114 (1993)Google Scholar
- 5.Brinkmann, A., Salzwedel, K., Scheideler, C.: Efficient, distributed data placement strategies for storage area networks. In: Proceedings of the 12th ACM Symposium on Parallel Algorithms and Architectures (SPAA), pp. 119–128. ACM Press, New York (2000) Extended AbstractGoogle Scholar
- 6.Brinkmann, A., Salzwedel, K., Scheideler, C.: Compact, adaptive placement schemes for non-uniform capacities. In: Proceedings of the 14th ACM Symposium on Parallel Algorithms and Architectures (SPAA), Winnipeg, Manitoba, Canada, August 2002, pp. 53–62 (2002)Google Scholar
- 7.Honicky, R.J., Miller, E.L.: A fast algorithm for online placement and reorganization of replicated data. In: Proceedings of the 17th International Parallel & Distributed Processing Symposium, Nice, France (April 2003)Google Scholar
- 8.Honicky, R.J., Miller, E.L.: Replication under scalable hashing: A family of algorithms for scalable decentralized data distribution. In: Proceedings of the 18th International Parallel & Distributed Processing Symposium (IPDPS 2004), Santa Fe, NM, April 2004. IEEE, Los Alamitos (2004)Google Scholar
- 9.Liu, Z., Zhou, X.-M.: An Adaptive Data Objects Placement Algorithm For Non-Uniform Capacities. In: Proceedings of the 3rd International Conference on Grid and Cooperative Computing, WuHan (October 2004)Google Scholar