HPCC 2006: High Performance Computing and Communications pp 783-792 | Cite as
Developing a Consistent Data Sharing Service over Grid Computing Environments
Abstract
Data replication is an important issue in computational grid environments where many data-intensive scientific applications require high-performance data accesses to remote storages. However, providing the consistent data replication service for computational grid environments is not an easy task because it requires a sophisticated technique to minimize I/O and communication latencies incurred during data copy steps to the remote clients. We have developed a replication architecture that is combined with data compression to reduce I/O and network overheads, with the help of data base. In this paper we describe the design and implementation of our replication architecture and present performance results on Linux clusters.
Preview
Unable to display preview. Download preview PDF.
References
- 1.Allcock, B., Foster, I., Nefedova, V., Chervenak, A., Deelman, E., Kesselman, C., Leigh, J., Sim, A., Shoshani, A., Drach, B., Williams, D.: High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies. In: SC 2001 (November 2001)Google Scholar
- 2.Moore, R., Rajasekar, A.: Data and Metadata Collections for Scientific Applications. In: High Performance Computing and Networking (HPCN 2001), Amsterdam, NL (June 2001)Google Scholar
- 3.Chervenak, A., Deelman, E., Kesselman, C., Pearlman, L., Singh, G.: A Metadata Catalog Service for Data Intensive Applications. GriPhyN technical report (2002)Google Scholar
- 4.Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications 23, 187–200 (2001)CrossRefGoogle Scholar
- 5.Chervenak, A., Schuler, R., Kesselman, C., Koranda, S., Moe, B.: Wide Area Data Replication for Scientific Collaborations. In: Proceedings of 6th IEEE/ACM International Workshop on Grid Computing (Grid 2005) (November 2005)Google Scholar
- 6.Thakur, R., Gropp, W.: Improving the Performance of Collective Operations in MPICH. In: Proceedings of the 10th European PVM/MPI Users’ Group Conference (Euro PVM/MPI 2003) (September 2003)Google Scholar