Frontiers of Computer Science

, Volume 8, Issue 3, pp 367–377

Hybrid hierarchy storage system in MilkyWay-2 supercomputer

  • Weixia Xu
  • Yutong Lu
  • Qiong Li
  • Enqiang Zhou
  • Zhenlong Song
  • Yong Dong
  • Wei Zhang
  • Dengping Wei
  • Xiaoming Zhang
  • Haitao Chen
  • Jianying Xing
  • Yuan Yuan
Research Article

DOI: 10.1007/s11704-014-3499-6

Cite this article as:
Xu, W., Lu, Y., Li, Q. et al. Front. Comput. Sci. (2014) 8: 367. doi:10.1007/s11704-014-3499-6

Abstract

With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability.MilkyWay-2 (TH-2) supercomputer system with peak performance 54.9 Pflops, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability.

Keywords

supercomputer storage system file system MilkyWay-2 hybrid hierarchy 

Copyright information

© Higher Education Press and Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Weixia Xu
    • 1
    • 2
  • Yutong Lu
    • 1
    • 2
  • Qiong Li
    • 2
  • Enqiang Zhou
    • 1
    • 2
  • Zhenlong Song
    • 2
  • Yong Dong
    • 1
    • 2
  • Wei Zhang
    • 1
    • 2
  • Dengping Wei
    • 2
  • Xiaoming Zhang
    • 2
  • Haitao Chen
    • 1
    • 2
  • Jianying Xing
    • 2
  • Yuan Yuan
    • 2
  1. 1.State Key Laboratory of High Performance ComputingChangshaChina
  2. 2.College of ComputerNational University of Defense TechnologyChangshaChina

Personalised recommendations