Abstract
Big data is an emerging term in the storage industry, and it is data analytics on big storage, i.e., Cloud-scale storage. In Cloud-scale storage systems, load balancing in request workloads across a metadata server cluster is critical for avoiding performance bottlenecks and improving quality of services. Many good approaches have been proposed for load balancing in distributed storage systems. Some of them pay attention to global namespace balancing, making metadata distribution across metadata servers as uniform as possible. However, they do not work well in skew request distributions, which impair load balancing but simultaneously increase the effectiveness of caching and replication. In this paper, we propose Cloud Cache (C 2), an adaptive load balancing scheme for metadata server cluster in Cloud-scale storage systems. It combines adaptive cache diffusion and replication scheme to cope with the request load balancing problem, and it can be integrated into existing distributed metadata management approaches to efficiently improve their load balancing performance. By conducting a performance evaluation in trace-driven simulations, experimental results demonstrate the efficiency and scalability of C 2.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Raicu, I., Foster, I.T., Beckman, P.: Making a case for distributed file systems at Exascale. In: LSAP, pp. 11–18 (2011)
Amer, A., Long, D., Schwarz, T.: Reliability Challenges for Storing Exabytes. In: International Conference on Computing, Networking and Communications (ICNC), CNC Workshop (2014)
Ousterhout, J.K., Costa, H.D., Harrison, D., Kunze, J.A., Kupfer, M.D., Thompson, J.G.: A trace-driven analysis of the UNIX 4.2 BSD file system. In: SOSP, pp. 15–24 (1985)
Xu, Q., Arumugam, R., Yong, K.L., Mahadevan, S.: Efficient and Scalable Metadata Management in EB-scale File Systems. IEEE Transactions on Parallel and Distributed Systems 99, 1 (2013) (PrePrints)
Kavalanekar, S., Worthington, B.L., Zhang, Q., Sharda, V.: Characterization of storage workload traces from production windows servers. In: Christie, D., Lee, A., Mutlu, O., Zorn, B.G. (eds.) 4th International Symposium on Workload Characterization (IISWC 2008, September 14-16, pp. 119–128. IEEE, Seattle (2008)
Ellard, D., Ledlie, J., Malkani, P., Seltzer, M.I.: Passive NFS tracing of email and research workloads. In: Chase, J. (ed.) Proceedings of the FAST 2003 Conference on File and Storage Technologies, March 31 - April 2. USENIX, Cathedral Hill Hotel (2003)
Xu, Q., Arumugam, R.V., Yang, K.L., Mahadevan, S.: DROP: Facilitating distributed metadata management in EB-scale storage systems. In: IEEE 29th Symposium on Mass Storage Systems and Technologies, MSST 2013, Long Beach, CA, USA, May 6-10, pp. 1–10 (2013)
Stoica, I., Morris, R., Karger, D.R., Kaashoek, M.F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for internet applications. In: SIGCOMM, pp. 149–160 (2001)
Ledlie, J., Seltzer, M.I.: Distributed, secure load balancing with skew, heterogeneity and churn. In: 24th Annual Joint Conference of the IEEE Computer and Communications Societies, INFOCOM 2005, March 13-17, pp. 1419–1430. IEEE, Miami (2005)
Gummadi, P.K., Dunn, R.J., Saroiu, S., Gribble, S.D., Levy, H.M., Zahorjan, J.: Measurement, modeling, and analysis of a peer-to-peer file-sharing workload. In: Scott, M.L., Peterson, L.L. (eds.) Proceedings of the 19th ACM Symposium on Operating Systems Principles 2003, SOSP 2003, October 19-22, pp. 314–329. ACM, Bolton Landing (2003)
Cai, Q., Arumugam, R.V., Xu, Q., He, B.: Understanding the Behavior of Solid State Disk. In: The 18th Asia Pacific Symposium on Intelligent and Evolutionary Systems (to appear, 2014)
Khuller, S., Kim, Y.A., Wan, Y.J.: Algorithms for data migration with cloning. In: Neven, F., Beeri, C., Milo, T. (eds.) Proceedings of the Twenty-Second ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, June 9-12, pp. 27–36. ACM, San Diego (2003)
Xu, Q., Shen, H.T., Cui, B., Hou, X., Dai, Y.: A novel content distribution mechanism in DHT networks. In: Fratta, L., Schulzrinne, H., Takahashi, Y., Spaniol, O. (eds.) NETWORKING 2009. LNCS, vol. 5550, pp. 742–755. Springer, Heidelberg (2009)
Ratnasamy, S., Handley, M., Karp, R.M., Shenker, S.: Topologically-aware overlay construction and server selection. In: INFOCOM (2002)
Xu, Q., Zhao, L., Xiao, M., Liu, A., Dai, Y.: YuruBackup: A Space-Efficient and Highly Scalable Incremental Backup System in the Cloud. International Journal of Parallel Programming, 1–23 (2013)
Cao, Y., Chen, C., Guo, F., Jiang, D., Lin, Y., Ooi, B.C., Vo, H.T., Wu, S., Xu, Q.: Es2: A cloud data storage system for supporting both OLTP and OLAP. In: Proceedings of the 27th International Conference on Data Engineering, ICDE 2011, April 11-16, pp. 291–302 (2011)
Naor, M., Wieder, U.: Novel architectures for P2P applications: The continuous-discrete approach. ACM Transactions on Algorithms 3(3) (2007)
Dabek, F., Kaashoek, M.F., Karger, D.R., Morris, R., Stoica, I.: Wide-Area Cooperative Storage with CFS. In: SOSP, pp. 202–215 (2001)
Gopalakrishnan, V., Silaghi, B.D., Bhattacharjee, B., Keleher, P.J.: Adaptive replication in peer-to-peer systems. In: 24th International Conference on Distributed Computing Systems (ICDCS 2004), March 24-26, pp. 360–369. IEEE Computer Society, Hachioji (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Xu, Q., Arumugam, R.V., Yong, K.L., Wen, Y., Ong, YS. (2015). C2: Adaptive Load Balancing for Metadata Server Cluster in Cloud-Scale Storage Systems. In: Handa, H., Ishibuchi, H., Ong, YS., Tan, K. (eds) Proceedings of the 18th Asia Pacific Symposium on Intelligent and Evolutionary Systems, Volume 1. Proceedings in Adaptation, Learning and Optimization, vol 1. Springer, Cham. https://doi.org/10.1007/978-3-319-13359-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-13359-1_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13358-4
Online ISBN: 978-3-319-13359-1
eBook Packages: EngineeringEngineering (R0)