Abstract
An algorithmic mapping of storage locations brings high storage efficiency to the storage system, but the loss of efficient scheduling makes systems prone to crashing at low usage. This paper uses the Ceph storage system as a research sample to analyze these issues and proposes a grading mapping adaptive storage resource collaborative optimization mechanism. This approach grading both the storage device and the storage content, and introduced random factors and influence factors as two-factors to quantify the grading mapping relationship between the two of them. This relation coordinates the storage systems’ performance and reliability. In addition, a collaborative storage algorithm is proposed to realize balanced storage efficiency and control data migration. The experimental results show that in comparison with the inherent mechanism in the traditional Ceph system, the proposed cooperative storage adaptation mechanism for data balancing has increased the average system usage by 17% and reduces data migration by 50% compared to the traditional research approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang, W., Flores, H., Pan, H.U.I.: Towards collaborative multi-device computing. In: 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE (2018)
Rump, F., Timm, B., Raphael, E.: Distributed and collaborative malware analysis with MASS. In: 2017 IEEE 42nd Conference on Local Computer Networks (LCN). IEEE (2017)
Aghayev, A., et al.: File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution. In: Proceedings of the 27th ACM Symposium on Operating Systems Principles. ACM (2019)
Zhou, J., et al.: Pattern-directed replication scheme for heterogeneous object-based storage. In: 17th IEEE/ACM International Symposium on Cluster 2017, Cloud and Grid Computing (CCGRID). IEEE (2017)
Kisley, R.V., Philip, D.K.: Distributed file serving architecture with metadata storage and data access at the data server connection speed. U.S. Patent No. 9,262,094. 16 February 2016
Huang, M., et al.: Research on data migration optimization of Ceph. In: 2017 14th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). IEEE (2017)
Zhao, N., et al.: A reliable power management scheme for consistent hashing based distributed key value storage systems. Front. Inf. Technol. Electron. Eng. 17(10), 994–1007 (2016)
Zhang, X., Gaddam, S., Chronopoulos, A.T.: Ceph distributed file system benchmarks on an openstack cloud. In: 2015 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM). IEEE (2015)
Wang, L.: Optimizations on Ceph Cache Tiering. KylinCloud, Ceph day (2015)
Weil, S.A., et al.: RADOS: a scalable, reliable storage service for petabyte-scale storage clusters. In: Proceedings of the 2nd International Workshop on Petascale Data Storage: Held in Conjunction with Supercomputing 2007. ACM (2007)
Zhou, J., et al.: Pattern-directed replication scheme for heterogeneous object-based storage. In: 17th IEEE/ACM International Symposium on Cluster 2017, Cloud and Grid Computing (CCGRID). IEEE (2017)
Mseddi, A., Salahuddin, M.A., Zhani, M.F., et al.: Efficient replica migration scheme for distributed cloud storage systems. IEEE Trans. Cloud Comput. (2018)
D’atri, A., Bhembre, V., Singh, K.: Learning Ceph: unifed, scalable, and reliable open source storage solution. Packt Publishing Ltd. (2017)
Ou, J., et al.: EDM: an endurance-aware data migration scheme for load balancing in SSD storage clusters. In: 2014 IEEE 28th International Parallel and Distributed Processing Symposium. IEEE (2014)
Zhang, X., Wang, Y., Wang, Q., et al.: A new approach to double I/O performance for Ceph distributed file system in cloud computing. In: 2019 2nd International Conference on Data Intelligence and Security (ICDIS), pp. 68–75. IEEE (2019)
Aghayev, A., Weil, S., Kuchnik, M., et al.: File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution. In: Proceedings of the 27th ACM Symposium on Operating Systems Principles, pp. 353–369 (2019)
Acknowledgment
This work is supported by the Natural Science Foundation of China(No. 61762008), the National Key Research and Development Project of China (No. 2018YFB1404404), the Major special project of science and technology of Guangxi(No.AA18118047-7), and the Guangxi Natural Science Foundation Project (No. 2017GXNSFAA198141).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Lu, Y., Chen, N., Pu, W., Wang, R. (2021). A New Collaborative Scheduling Mechanism Based on Grading Mapping for Resource Balance in Distributed Object Cloud Storage System. In: Gao, H., Wang, X., Iqbal, M., Yin, Y., Yin, J., Gu, N. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 350. Springer, Cham. https://doi.org/10.1007/978-3-030-67540-0_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-67540-0_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67539-4
Online ISBN: 978-3-030-67540-0
eBook Packages: Computer ScienceComputer Science (R0)