Abstract
Load balance is critical for large-scale storage systems to produce high I/O performance. Decentralized solutions are especially preferred for no single point of bottleneck. We implement four typical hypercube-based decentralized load balancing algorithms in a prototype storage system, and conduct extensive experiments with the system running on a testbed comprising 32 nodes. We compare the efficiency and scalability of the four algorithms through the experiments. The comparison results lead to the following new observations contrary to the conclusions obtained in previous simulation studies. Firstly, algorithms with no redundant load migration do not actually achieve savings of migration costs. Secondly, algorithms tolerating a certain degree of redundancy in load migration may achieve improvements in scalability. The two observations provide new insights into the design of load balancing algorithms in distributed storage systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Maccormick, J., Murphy, N., Ramasubramanian, V., Wieder, U., Yang, J., Zhou, L.: Kinesis: A New Approach to Replica Placement in Distributed Storage Systems. ACM Transactions on Storage 4(4), 11:1–11:28 (2009)
Kari, C., Kim, Y.A., Russell, A.: Data Migration in Heterogeneous Storage Systems. In: 31st IEEE International Conference on Distributed Computing Systems, pp. 143–150. IEEE Computer Society, Minneapolis (2011)
Wei, Q.: CDRM: a Cost-effective Dynamic Replication Management Scheme for Cloud Storage Cluster. In: IEEE International Conference on Cluster Computing, pp. 188–196. IEEE Computer Society, Heraklion (2010)
Xie, C., Cai, B.: A Decentralized Storage Cluster with High Reliability and flexibility. In: 14th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, IEEE Computer Society, Montbeliard-Sochaux (2006)
Ranka, S., Won, Y., Sahni, S.: Programming a hypercube multicomputer. IEEE Software 5(5), 69–77 (1998)
Nicol, D.M.: Communication efficient global load balancing. In: International Conference on Scalable High Performance Computing, pp. 292–299. IEEE (1992)
Luo, X., Wang, Y.: DBDS: a Fully Distributed Algorithm for Data Migration. Computer Applications and Software 28(11), 45–48 (2011) (in China)
Wu, M.Y., Shu, W.: A Load-Balancing Algorithm for n-cubes. In: International Conference on Parallel Processing, pp. 148–155. IEEE (1996)
Dowdy, W., Foster, D.: Comparative Models of the File Assignment Problem. ACM Computing Surveys 14(2), 287–313 (1982)
Graham, R.L.: Bounds on Multiprocessing Timing Anomalies. SIAM Journal on Applied Mathematics 17(2), 416–429 (1969)
Lee, L.W., Scheuermann, P., Vingralek, R.: File Assignment in Parallel I/O Systems with Minimal Variance of Service Time. IEEE Transactions on Computers 49(2), 127–140 (2000)
Madathil, D.K., Thota, R.B., Paul, P., Xie, T.: A Static Data Placement Strategy towards Perfect Load-Balancing for Distributed Storage Clusters. In: International Symposium on Parallel and Distributed Processing, pp. 1–8. IEEE Computer Society, Miami (2008)
Lumb, C.R., Golding, R., Ganger, G.R.: D-SPTF: Decentralized Request Distribution in Brick based Storage Systems. In: the 11th International Conference on Architectural Support for Programming Languages and Operating, pp. 37–47. ACM, Boston (2004)
Ghemawat, S., Gobioff, H., Leung, S.T.: The Google File System. In: 19th ACM Symposium on Operating Systems Principles, pp. 29–43. ACM, Bolton Landing (2003)
Hall, J., Hartline, J., Karlin, A.R., Saia, J., Wilkes, J.: On Algorithms for Efficient Data Migration. In: 12th Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 620–629. ACM, Washington, DC (2001)
Bodik, P.: Automating Datacenter Operations Using Machine Learning. Doctoral Dissertation, University of California, Berkeley (2010)
Wang, W., Zhao, Y.: A Novel Network Storage Scheme: Intelligent Network Disk Storage Cluster. In: 5th International Conference on Networking, Sensing and Control, pp. 142–147. IEEE Computer Society, Sanya (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, Y., Luo, X., Yuan, F., Li, C. (2013). An Empirical Comparative Study of Decentralized Load Balancing Algorithms in Clustered Storage Environment. In: Zu, Q., Hu, B., Elçi, A. (eds) Pervasive Computing and the Networked World. ICPCA/SWS 2012. Lecture Notes in Computer Science, vol 7719. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37015-1_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-37015-1_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37014-4
Online ISBN: 978-3-642-37015-1
eBook Packages: Computer ScienceComputer Science (R0)