Grid-Based Processing of High-Volume Meteorological Data Sets
Our energy production increasingly depends on regenerative energy sources, which impose new challenges. One problem is the availability of regenerative energy sources like wind and solar radiation that is influenced by fluctuating meteorological conditions. Thus the development of forecast methods capable of determining the level of power generation (e.g., through wind or solar power) in near real-time is needed. Another scenario is the determination of optimal locations for power plants. These aspects are considered in the domain of energy meteorology. For that purpose large data repositories from many heterogeneous sources (e.g., satellites, earth stations, and data archives) are the base for complex computations. The idea is to parallelize these computations in order to obtain significant speed-ups. This paper reports on employing Grid technologies within an ongoing project, which aims to set up a Grid infrastructure among several geographically distributed project partners. An approach to transfer large data sets from many heterogenous data sources and a means of utilizing parallelization are presented. For this purpose we are evaluating various Grid middleware platforms. In this paper we report on our experience with Globus Toolkit 4, Condor, and our first experiments with UNICORE.
KeywordsData Transfer Message Passing Interface Grid Service Project Partner Grid Infrastructure
Unable to display preview. Download preview PDF.
- 1.Foster, I.: What is the Grid? A Three Point Checklist. Grid Today 1(6) (2002), http://www.gridtoday.com/02/0722/100136.html
- 3.MPI: Message Passing Interface. http://www-unix.mcs.anl.gov/mpi/standard.html. Retrieved: 2006-06-11
- 4.Globus: The Globus Alliance. http://www.globus.org/. Retrieved: 2006-06-11.
- 5.Globus: Globus Toolkit 4.0 Release Manuals. http://www.globus.org/toolkit/docs/4.0/. Retrieved: 2006-06-11
- 6.Jacob, B., Brown, M., Fukui, K., Trivedi, N.: Introduction to Grid Computing. http://www.redbooks.ibm.com/redbooks/pdfs/sg246895.pdf. Retrieved: 2006-06-11
- 7.WSRF: Web Services Resource Framework. http://www.globus.org/wsrf/. Retrieved: 2006-06-11
- 8.University of Wisconsin: Condor High Throughput Computing. http://www.cs.wisc.edu/condor/. Retrieved: 2006-06-11
- 9.Mayer, B.: I3RC phase 1 results from the MYSTIC Monte Carlo model. In: I3RC (Intercomparison of 3D Radiation Codes) workshop, Tucson, Arizona (1999)Google Scholar
- 10.Mayer, B., Kylling, A., Hamann, U.: libRadtran – library for radiative transfer. http://www.libradtran.org. Retrieved: 2006-06-11
- 11.Geist, A.: PVM: Parallel Virtual Machine: A Users’ Guide and Tutorial for Network Parallel Computing. Scientific and Engineering Computation. MIT Press, Cambridge (1994)Google Scholar
- 12.EGEE: gLite Lightweight Middleware for Grid Computing. http://glite.web.cern.ch/glite/. Retrieved: 2006-06-11
- 13.Sun: Sun N1 Grid Engine. http://www.sun.com/software/gridware/. Retrieved: 2006-06-11
- 14.OGSA-DAI: Open Grid Services Architecture - Data Access and integration. http://www.ogsadai.org.uk/. Retrieved: 2006-06-11
- 15.Hasselbring, W., Reussner, R.: Toward trustworthy software systems. IEEE Computer 39(4), 91–92 (2006)Google Scholar