Advertisement

The Journal of Supercomputing

, Volume 73, Issue 5, pp 2052–2068 | Cite as

A hybrid disaster-tolerant model with DDF technology for MooseFS open-source distributed file system

  • Xigao Li
  • Lin Qian
Article

Abstract

In a distributed file system (DFS), a disaster-tolerant model provides high data security in the data disaster event by geographically isolates the backup data, while the hot-backup model provides high availability when the main server is offline. Therefore, the combined model is widely concerned in multiple fields. However, previous investigations have neglected to the disaster tolerance for open-source distributed file systems. When a server run into a fault status and performs backup switch, the remote data center cannot receive any data replicas. We proposed a hybrid model in this work to provide disaster-tolerance especially for open-source DFS by build geographically isolate replicas. In addition, we optimize the model with direct data fetch (DDF) technology. DDF can bypass the main server, and retrieve data directly from the data node. The model was implemented on an open-source DFS named MooseFS. Compared with HDFS and IBM/PPRC, the hybrid model with DDF provides a strong performance boost to the hybrid model during the switch procedure, and this hybrid model performs best for storage of large amount of files in small size.

Keywords

MooseFS Disaster tolerant Direct data fetch Distributed file system Hybrid model 

References

  1. 1.
    Manchester J, Saha D, Tripathi SK (2004) Guest editorial-protection, restoration, and disaster recovery. IEEE Netw 18(2):3–4CrossRefGoogle Scholar
  2. 2.
    Ceph File System Docs (2016) http://docs.ceph.com/docs/master/cephfs/#ceph-filesystem. Accessed 8 Apr 2016
  3. 3.
    Moon YH, Youn CH (2015) Multihybrid job scheduling for fault-tolerant distributed computing in policy-constrained resource networks[J]. Comput Netw 82:81–95CrossRefGoogle Scholar
  4. 4.
    Yanlong W (2008) Research of a soft-synchronous replication algorithm for disaster tolerance. Chin High Technol LettGoogle Scholar
  5. 5.
    Chang F, Ji M, Leung S et al (2002) Myriad cost-effective disaster tolerance. In: Proceeedings of Fast. USENIX Association, pp 103–116Google Scholar
  6. 6.
    Taobao File System (2011) http://code.taobao.org/p/tfs/wiki/intro/. Accessed 12 Aug 2015
  7. 7.
    Deng K, Wang K, Ma D (2016) A new solution of distributed disaster recovery based on raptor code. In: Proceedings of the 2015 International Conference on Applied Mechanics, Mechatronics and Intelligent Systems (AMMIS2015), pp 825–832Google Scholar
  8. 8.
    Ghemawat S, Gobioff H, Leung S (2004) The Google file system. Proc ACM Sigmetrics Int Conf Meas Model Comput Syst ACM 37:29-43Google Scholar
  9. 9.
    Li-jiang P, Yong-tang B (2008) The analysis and application of Oracle DataGuard for Diff-Area disaster recovery. Comput Knowl Technol 2008:28Google Scholar
  10. 10.
    Massiglia P (2000) VERITAS volume replication and Oracle databases. VERITAS Corporation, Mountain ViewGoogle Scholar
  11. 11.
    Liu X-J (2010) A brief analysis of the disaster recovery backup technology in oracle database dataguard. In: 2010 2nd International Conference on Industrial and Information Systems, IIS, vol 2, pp 234–236Google Scholar
  12. 12.
    Brown TM, Lipets ML (2014) Metadata management. US20140344526, IEEEGoogle Scholar
  13. 13.
    Borthakur D (2007) The hadoop distributed file system: architecture and design. Hadoop Project Website 2007(11):21Google Scholar
  14. 14.
    Depardon B, Mahec GL, Séguin C (2013) Analysis of six distributed file systems. HAL-INRIA 23(4):525–542Google Scholar
  15. 15.
    Yunxiao A, Yuesheng T, Jingyu W (2013) Chunkserver load balancing selection algorithm on MooseFS. In: Microcomputer & Its ApplicationsGoogle Scholar
  16. 16.
    Rajasekar A, Moore R, Hou Cy, Lee CA, Marciano R, de Torcy A, Wan M, Schroeder W, Chen SY, Gilbert L, Tooby P, Zhu B (2010) iRODS primer: integrated rule-oriented data system. Morgan and Claypool Publishers, San RafaelMATHGoogle Scholar
  17. 17.
    Wang T, Gao H, Qiu J (2016) A combined fault-tolerant and predictive control for network-based industrial processes. IEEE Trans Industr Electron 63(4):2529–2536Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.State Grid Electric Power Research InstituteNanjingChina

Personalised recommendations