Applications of Virtual Data in the LIGO Experiment
Many Physics experiments today generate large volumes of data. That data is then processed in many ways in order to achieve the understanding of fundamental physical phenomena. Virtual Data is a concept that unifies the view of the data whether it is raw or derived. It provides a new degree of transparency in how data-handling and processing capabilities are integrated to deliver data products to end-users or applications, so that requests for such products are easily mapped into computation and/or data access at multiple locations. GriPhyN (Grid Physics Network) is a NSF-funded project, which aims to realize the concepts of Virtual Data. Among the physics applications participating in the project is the Laser Interferometer Gravitational-wave Observatory (LIGO), which is being built to observe the gravitational waves predicted by general relativity. LIGO will produce large amounts of data, which are expected to reach hundreds of petabytes over the next decade. Large communities of scientists, distributed around the world, need to access parts of these datasets and perform efficient analysis on them. It is expected that the raw and processed data will be distributed among various national centers, university computing centers, and individual workstations. In this paper we describe some of the challenges associated with building Virtual Data Grids for experiments such as LIGO.
KeywordsGravitational Wave Directed Acyclic Graph Compact Muon Solenoid Data Grid Grid Environment
Unable to display preview. Download preview PDF.
- 1.W. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke. ”data management and transfer in high-performance computational grid environments.”. Parallel Computing, 2001.Google Scholar
- 2.W. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke. ”secure, efficient data transport and replica management for high-performance data-intensive computing.”. In IEEE Mass Storage Conference, 2001.Google Scholar
- 3.W. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, A. Sim, A. Shoshani, B. Drach, and D. Williams. ”high-performance remote access to climate simulation data: A challenge problem for data grid technologies”. 2001. SC’2001.Google Scholar
- 4.B. Allen, E. Deelman, C. Kesselman, A. Lazzarini, T. Prince, J. Romano, and R. Williams. ”ligo’s virtual data requirements.”. Technical Report 6, GriPhyN, 2001.Google Scholar
- 5.O. Babaoglu and K. Marzullo. Consistent global states of distributed systems: Fundamental concepts and mechanisms. Technical Report UBLCS-93-1, University of Bologna, 1983.Google Scholar
- 6.K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith, and S. Tuecke. A resource management architecture for metacomputing systems. In The 4th Workshop on Job Scheduling Strategies for Parallel Processing, pages 62–82, 1998.Google Scholar
- 7.Karl Czajkowski, Steven Fitzgerald, Ian Foster, and Carl Kesselman. Grid information services for distributed resource sharing. In Proc. 10th IEEE Symp. on High Performance Distributed Computing, 2001.Google Scholar
- 8.E. Deelman, C. Kesselman, and G. Mehta. ”transformation catalog design for griphyn”. Technical Report 17, GriPhyN, 2001.Google Scholar
- 9.E. Deelman, C. Kesselman, R. Williams, A. Lazzarini, T. A. Prince, J. Romano, and B. Allen. ”a virtual data grid for ligo”. volume 2110 of Lecture Notes in Computer Science, pages 3–12, 2001.Google Scholar
- 11.I. Foster and C. Kesselman. The Globus project: A status report. In Proceedings of the Heterogeneous Computing Workshop, pages 4–18. IEEE Computer Society Press, 1998.Google Scholar
- 12.I. Foster and C. Kesselman, editors. The Grid: Blueprint for a Future Computing Infrastructure. 1999.Google Scholar
- 13.I. Foster, C. Kesselman, G. Tsudik, and S. Tuecke. A security architecture for computational grids. In ACM Conference on Computers and Security, pages 83–91. ACM Press, 1998.Google Scholar
- 14.I. Foster, C. Kesselman, and S. Tuecke. ”the anatomy of the grid: Enabling scalable virtual organizations.”. Intl. J. Supercomputer Applications, 15(3), 2001.Google Scholar
- 15.J. Frey, T. Tannenbaum, I. Foster, M. Livny, and S. Tuecke. ”condor-g: A computation management agent for multi-institutional grids”. In Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10), 2001.Google Scholar
- 16.”MetaNEOS Project”. http://www-unix.mcs.anl.gov/metaneos/nug30/pr.html, 2001.
- 17.R. Wolski. Forecasting network performance to support dynamic scheduling using the network weather service. In Proc. 6th IEEE Symp. on High Performance Distributed Computing, Portland, Oregon, 1997. IEEE Press.Google Scholar