Abstract
Data-intensive applications require the efficient management and transfer of terabytes or petabytes of information in wide-area. Data management system in distributed computing environment could not meet this requirement efficiently. Storage visualization is an efficient way to solve these problems. It integrates all kinds of high performance storage system to a unit one. It can not only share resources and make fully use of resources, but efficiently avoid conflict of data explosion and limited storage ability. To resolve these conflicts, we propose a Global Distributed Storage System (GDSS).
With data explosion, metadata becomes more and more enormous. Exinting methods could not keep up with the need of scalability, availability and efficiency to manage metadata. To solve these problems, MDC (MetaData Controller), a novel metadata management scheme based on MatchTable, is introduced in GDSS. In MDC, MatchTable is responsible for communication between SSP (Storage Service Point) and DS (Directory Server), and cache module is responsible for increasing of metadata access efficiency and as an assistant to maintain the replica coherence of metadata. MDC locates metadata only once by matching MatchTable, which avoids searching it from root directory server step by step. Experiment results indicate that MDC can locate metadata fast and supply fault tolerance, and also has better availability and scalability.
This paper is supported by National Science Foundation under grant 60125208 and 60273076, ChinaGrid project from Ministry of Education, and National 973 Key Basic Research Program under grant No.2003CB317003
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jin H., Ran L., Wang Z., et al.: Architecture Design of Global Distributed Storage System for Data Grid. High Technology Lett., 9(4), 1–4 (2003)
Han H., Guo C., Dai Y., et al.: A scheme to construct global file system. In: Proceedings of the Second International Conference on Web Information Systems Engineering. IEEE, Inc. Kyoto (2001),pp.206–212
Ellert M., Konstantinov A., Konya B., et al.: The NorduGrid project: Using Globus toolkit for building GRID infrastructure. Nuclear Instruments and Methods in Physics Research, Section A. Elsevier 502, 407–410 (2003)
Vazhkudai S., Tuecke S., Foster I.: Replica Selection in the Globus Data Grid. In: Proceedings of the 1st International Symposium on Cluster Computing and the Grid. IEEE, Inc. Brisbane (2001),pp.106–113
Bester J., Foster I., Kesselman C., et al.: GASS: A data movement and access service for wide area computing systems. In: Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems. ACM Press Atlanta (1999),pp.78–88
Thulasidasan S., Feng W.C., Gardner M.K: Optimizing GridFTP through Dynamic Right-Sizing. In: Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing. IEEE, Inc. Seattle (2003),pp.14–23
Rajasekar A., Wan M., Moore R.: MySRB and SRB—components of a Data Grid. In: Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing. IEEE, Inc. Edinburgh (2002),pp.301–310
Wan M., Rajasekar A., Moore R., et al.: A simple mass storage system for the SRB data grid. In: Proceedings of 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies. IEEE, Inc. San Diego (2003),pp.20–25
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yi, C., Jin, H., Jia, Y. (2005). Metadata Management in Global Distributed Storage System. In: Zhang, W., Tong, W., Chen, Z., Glowinski, R. (eds) Current Trends in High Performance Computing and Its Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-27912-1_17
Download citation
DOI: https://doi.org/10.1007/3-540-27912-1_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25785-1
Online ISBN: 978-3-540-27912-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)