Abstract
In a distributed file system (DFS), a disaster-tolerant model provides high data security in the data disaster event by geographically isolates the backup data, while the hot-backup model provides high availability when the main server is offline. Therefore, the combined model is widely concerned in multiple fields. However, previous investigations have neglected to the disaster tolerance for open-source distributed file systems. When a server run into a fault status and performs backup switch, the remote data center cannot receive any data replicas. We proposed a hybrid model in this work to provide disaster-tolerance especially for open-source DFS by build geographically isolate replicas. In addition, we optimize the model with direct data fetch (DDF) technology. DDF can bypass the main server, and retrieve data directly from the data node. The model was implemented on an open-source DFS named MooseFS. Compared with HDFS and IBM/PPRC, the hybrid model with DDF provides a strong performance boost to the hybrid model during the switch procedure, and this hybrid model performs best for storage of large amount of files in small size.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Manchester J, Saha D, Tripathi SK (2004) Guest editorial-protection, restoration, and disaster recovery. IEEE Netw 18(2):3–4
Ceph File System Docs (2016) http://docs.ceph.com/docs/master/cephfs/#ceph-filesystem. Accessed 8 Apr 2016
Moon YH, Youn CH (2015) Multihybrid job scheduling for fault-tolerant distributed computing in policy-constrained resource networks[J]. Comput Netw 82:81–95
Yanlong W (2008) Research of a soft-synchronous replication algorithm for disaster tolerance. Chin High Technol Lett
Chang F, Ji M, Leung S et al (2002) Myriad cost-effective disaster tolerance. In: Proceeedings of Fast. USENIX Association, pp 103–116
Taobao File System (2011) http://code.taobao.org/p/tfs/wiki/intro/. Accessed 12 Aug 2015
Deng K, Wang K, Ma D (2016) A new solution of distributed disaster recovery based on raptor code. In: Proceedings of the 2015 International Conference on Applied Mechanics, Mechatronics and Intelligent Systems (AMMIS2015), pp 825–832
Ghemawat S, Gobioff H, Leung S (2004) The Google file system. Proc ACM Sigmetrics Int Conf Meas Model Comput Syst ACM 37:29-43
Li-jiang P, Yong-tang B (2008) The analysis and application of Oracle DataGuard for Diff-Area disaster recovery. Comput Knowl Technol 2008:28
Massiglia P (2000) VERITAS volume replication and Oracle databases. VERITAS Corporation, Mountain View
Liu X-J (2010) A brief analysis of the disaster recovery backup technology in oracle database dataguard. In: 2010 2nd International Conference on Industrial and Information Systems, IIS, vol 2, pp 234–236
Brown TM, Lipets ML (2014) Metadata management. US20140344526, IEEE
Borthakur D (2007) The hadoop distributed file system: architecture and design. Hadoop Project Website 2007(11):21
Depardon B, Mahec GL, Séguin C (2013) Analysis of six distributed file systems. HAL-INRIA 23(4):525–542
Yunxiao A, Yuesheng T, Jingyu W (2013) Chunkserver load balancing selection algorithm on MooseFS. In: Microcomputer & Its Applications
Rajasekar A, Moore R, Hou Cy, Lee CA, Marciano R, de Torcy A, Wan M, Schroeder W, Chen SY, Gilbert L, Tooby P, Zhu B (2010) iRODS primer: integrated rule-oriented data system. Morgan and Claypool Publishers, San Rafael
Wang T, Gao H, Qiu J (2016) A combined fault-tolerant and predictive control for network-based industrial processes. IEEE Trans Industr Electron 63(4):2529–2536
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, X., Qian, L. A hybrid disaster-tolerant model with DDF technology for MooseFS open-source distributed file system. J Supercomput 73, 2052–2068 (2017). https://doi.org/10.1007/s11227-016-1902-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-016-1902-9