Frontiers of Computer Science

, Volume 11, Issue 1, pp 75–87 | Cite as

HWM: a hybrid workload migration mechanism of metadata server cluster in data center

  • Jian Liu
  • Huanqing Dong
  • Junwei Zhang
  • Zhenjun Liu
  • Lu XuEmail author
Research Article


In data center, applications of big data analytics pose a big challenge to massive storage systems. It is significant to achieve high availability, high performance and high scalability for PB-scale or EB-scale storage systems. Metadata server (MDS) cluster architecture is one of the most effective solutions to meet the requirements of applications in data center. Workload migration can achieve load balance and energy saving of cluster systems. In this paper, a hybrid workload migration mechanism of MDS cluster is proposed and named as HWM. In HWM, workload of MDS is classified into two categories: metadata service and state service, and they can be migrated rapidly from a source MDS to a target MDS in different ways. Firstly, in metadata service migration, all the dirty metadata of one sub file system is flushed to a shared storage pool by the source MDS, and then is loaded by the target MDS. Secondly, in state service migration, all the states of that sub file system are migrated from source MDS to target MDS through network at file granularity, and then all of the related structures of these states are reconstructed in targetMDS. Thirdly, in the process of workload migration, instead of blocking client requests, the source MDS can decide which MDS will respond to each request according to the operation type and the migration stage. The proposed mechanismis implemented in the BlueWhaleMDS cluster. The performance measurements show that the HWM mechanism is efficient to migrate the workload of a MDS cluster system and provides low-latency access to metadata and states.


data center metadata server cluster hybrid workload migration metadata service state service lowlatency access 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This work was partially supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDA06010401), and the Tianjin Science and Technology Program (15ZXDSGX00020).

Supplementary material

11704_2016_6036_MOESM1_ESM.ppt (133 kb)
Supplementary material, approximately 133 KB.


  1. 1.
    Turner V, Gantz J F, Reinsel D, Minton S. The digital universe of opportunities: rich data and the increasing value of the Internet of things. IDC Analyze the Future, 2014Google Scholar
  2. 2.
    Ghemawat S, Gobioff H, Leung S T. The Google file system. ACM SIGOPS Operating Systems Review, 2003, 37(5): 29–43CrossRefGoogle Scholar
  3. 3.
    McKusick K, Quinlan S. GFS: evolution on fast-forward. Communications of the ACM, 2010, 53(3): 42–49CrossRefGoogle Scholar
  4. 4.
    Makoto S, Hiroki K, Shoji K. Performance evaluation of scaleout NAS for HDFS. In: Proceedings of the 3rd International Conference on Advances in Information Mining and Management. 2013, 32–35Google Scholar
  5. 5.
    Xia M, Saxena M, Blaum M, Pease D A. A tale of two erasure codes in HDFS. In: Proceedings of the 13th USENIX Conference on File and Storage Technologies. 2015, 213–226Google Scholar
  6. 6.
    Jain R, Sarkar P, Subhraveti D. GPFS-SNC: an enterprise cluster file system for big data. IBM Journal of Research and Development, 2013, 57(3/4): 5:1–5:10CrossRefGoogle Scholar
  7. 7.
    Davies A, Orsaria A. Scale out with GlusterFS. Linux Journal, 2013, (235): 1Google Scholar
  8. 8.
    Chasapis K, Dolz M F, Kuhn M, Ludwig T. Evaluating Lustre’s metadata server on a multi-socket platform. In: Proceedings of the 9th Parallel Data Storage Workshop. 2014, 13–18Google Scholar
  9. 9.
    Kim T, Noh S H. PNFS for everyone: an empirical study of a low-cost, highly scalable networked storage. International Journal of Computer Science and Network Security, 2014, 14(3): 52–59Google Scholar
  10. 10.
    Weil S A, Brandt S A, Miller E L, Long D D, Maltzahn C. Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation. 2006, 307–320Google Scholar
  11. 11.
    Wang F, Nelson M, Oral S, Atchley S, Weil S, Settlemyer B W, Caldwell B, Hill J. Performance and scalability evaluation of the Ceph parallel file system. In: Proceedings of the 8th Parallel Data StorageWorkshop. 2013, 14–19Google Scholar
  12. 12.
    Sevilla M A, Watkins N, Maltzahn C, Nassi I, Brandt S A, Weil S A, Farnum G, Fineberg S. Mantle: a programmable metadata load balancer for the Ceph file system. In: Proceedings of the 27th International Conference for High Performance Computing, Networking, Storage and Analysis. 2015, 1–12CrossRefGoogle Scholar
  13. 13.
    Sinnamohideen S, Sambasivan R R, Hendricks J, Liu L, Ganger G R. A transparently-scalable metadata service for the UrsaMinor storage system. In: Proceedings of USENIX Annual Technical Conference. 2010, 13–26Google Scholar
  14. 14.
    Abd-El-Malek M, Courtright IIWV, Cranor C, Ganger G R, Hendricks J, Klosterman A J, Mesnier M P, Prasad M, Salmon B, Sambasivan R R, S Sinnamohideen, Strunk J D, Thereska E, Wachs M, Wylie J J. Ursa Minor: versatile cluster-based storage. In: Proceedings of the 4th USENIX Conference on File and Storage Technologies. 2005, 59–72Google Scholar
  15. 15.
    Menon J, Pease D A, Rees R, Duyanovich L, Hillsberg B. IBM Storage Tank—a heterogeneous scalable SAN file system. IBM Systems Journal, 2003, 42(2): 250–267CrossRefGoogle Scholar
  16. 16.
    Thomasian A. Storage research in industry and universities. ACM SIGARCH Computer Architecture News, 2010, 38(2): 1–48CrossRefGoogle Scholar
  17. 17.
    An overview of NFSv4: NFSv4.0, NFSv4.1, pNFS, and proposed NFSv4.2 features. SNIA Ethernet Storage Forum, 2012Google Scholar
  18. 18.
    Mohr R, Peltz P. Benchmarking SSD-based Lustre file system configurations. In: Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment. 2014Google Scholar
  19. 19.
    Aishwarya K, Sreevatson M, Babu C, Prabavathy B. Efficient prefetching technique for storage of heterogeneous small files in hadoop distributed file system federation. In: Proceedings of the 15th International Conference on Advanced Computing. 2013, 523–530Google Scholar
  20. 20.
    Chen G, Jagadish H, Jiang D, Maier D, Ooi B C, Tan K L, Tan W C. Federation in cloud data management: challenges and opportunities. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1670–1678CrossRefGoogle Scholar
  21. 21.
    Patil S, Gibson G A. Scale and concurrency of GIGA+: file system directories with millions of files. In: Proceedings of the 9th USENIX Conference on File and Storage Technologies. 2011, 1–14Google Scholar
  22. 22.
    Patil S V, Gibson G A, Lang S, Polte M. GIGA+: scalable directories for shared file systems. In: Proceedings of the 2nd International Workshop on Petascale Data Storage. 2007, 26–29Google Scholar
  23. 23.
    Douceur J R, Howell J. Distributed directory service in the Farsite file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation. 2006, 321–334Google Scholar
  24. 24.
    Ma H, Liu Z, Zhang H, Feng S, Han X, Xu L. Experiences with hierarchical storage management support in blue whale file system. In: Proceedings of the 11th International Conference on Parallel and Distributed Computing, Applications and Technologies. 2010, 369–374Google Scholar
  25. 25.
    Solar R, Gil-Costa V, Marin M. Dynamic load balance for approximate parallel simulations with consistent hashing. In: Proceedings of the 47th Summer Computer Simulation Conference. 2015, 1–10Google Scholar
  26. 26.
    Xu Z Y, Wang X X. A predictive modified round robin scheduling algorithm for Web server clusters. In: Proceedings of the 34th Chinese Control Conference. 2015, 5804–5808Google Scholar
  27. 27.
    Xia Y, Dobra A, Han S C. Multiple-choice random network for server load balancing. In: Proceedings of the 26th IEEE International Conference on Computer Communications. 2007, 1982–1990Google Scholar
  28. 28.
    Wu Y, Luo S, Li Q. An adaptive weighted least-load balancing algorithm based on server cluster. In: Proceedings of the 5th International Conference on Intelligent Human-Machine Systems and Cybernetics. 2013, 224–227Google Scholar
  29. 29.
    Allayear S M, Salahuddin M, Ahmed F, Park S S. Introducing iSCSI protocol on online based mapreduce mechanism. In: Proceedings of the 14th International Conference on Computational Science and Its Applications. 2014, 691–706Google Scholar
  30. 30.
    Guo T, Shen Y L, Liu Z J, Xu L. BW-FILERAID: a kind of file based distributed RAID system and optimization. Applied Mechanics and Materials, 2011, 80: 1208–1216CrossRefGoogle Scholar
  31. 31.
    Chen Y, Wu F, Yu K, Zhang L, Chen Y, Yang Y, Mao J. Instant bug testing service for linux kernel. In: Proceedings of the 10th IEEE International Conference on High Performance Computing and Communications. 2013, 1860–1865Google Scholar

Copyright information

© Higher Education Press and Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Jian Liu
    • 1
    • 2
  • Huanqing Dong
    • 1
  • Junwei Zhang
    • 1
  • Zhenjun Liu
    • 1
  • Lu Xu
    • 1
    Email author
  1. 1.Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina

Personalised recommendations