Abstract
Nowadays, replication technique is widely used in data center storage systems to prevent data loss. Data popularity is a key factor in data replication as popular files are accessed most frequently and then they become unstable and unpredictable. Moreover, replicas placement is one of key issues that affect the performance of the system such as load balancing, data locality etc. Data locality is a fundamental problem to data-parallel applications that often happens and this problem leads to the decrease in performance. To address these challenges, this paper proposes a dynamic replication management scheme based on data popularity and data locality; it includes replica allocation and replica placement algorithms. Data locality, disk bandwidth, CPU processing speed and storage utilization are considered in the proposed data placement algorithm in order to achieve better data locality and load balancing effectively. Our proposed scheme will be effective for large-scale cloud storage.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hunger, A., Myint, J.: Comparative analysis of adaptive file replication algorithms for cloud data storage. In: 2014 International Conference on Future Internet of Things and Cloud (2014)
Gong, B., Veeravalli, B., Feng, D., Zeng, L., Wei, Q.: CDRM: a cost-effective dynamic replication management scheme for cloud storage cluster. In: 2010 IEEE International Conference on Cluster Computing, September 2010, pp. 188–196 (2010)
Abad, C.L., Lu, Y., Campbell, R.H.: DARE: adaptive data replication for efficient cluster scheduling. In: IEEE International Conference on Cluster Computing (CLUSTER 2011), pp. 159–168 (2011)
Lee, D., Lee, J., Chung, J.: Efficient data replication scheme based on hadoop distributed file system. Int. J. Softw. Eng. Appl. 9(12), 177–186 (2015)
Ananthanarayanan, G., et al.: Scarlett: coping with skewed content popularity in mapreduce clusters. In: Proceedings of Conference on Computer. Systems (EuroSys), pp. 287–300 (2011)
Gobioff, H., Ghemawat, S., Leung, S.-T.: The Google file system. In: Proceedings of 19th ACM Symposium on Operating Systems Principles (SOSP 2003), New York, USA, October 2003
Chang, H.-P., Chang, R.-S., Wang, Y.-T.: A dynamic weighted data replication strategy in data grids. In: 2008 IEEE/ACS International Conference on Computer Systems and Applications, March 2008, pp. 414–421 (2008)
Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., Stoica, I.: Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In: Proceeding of European Conference Computer System (EuroSys) (2010)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Thu, M.P., Nwe, K.M., Aye, K.N. (2019). Dynamic Replication Management Scheme for Distributed File System. In: Zin, T., Lin, JW. (eds) Big Data Analysis and Deep Learning Applications. ICBDL 2018. Advances in Intelligent Systems and Computing, vol 744. Springer, Singapore. https://doi.org/10.1007/978-981-13-0869-7_16
Download citation
DOI: https://doi.org/10.1007/978-981-13-0869-7_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-0868-0
Online ISBN: 978-981-13-0869-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)