Grid-Based Processing of High-Volume Meteorological Data Sets

  • Guido Scherp
  • Jan Ploski
  • Wilhelm Hasselbring
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4375)

Abstract

Our energy production increasingly depends on regenerative energy sources, which impose new challenges. One problem is the availability of regenerative energy sources like wind and solar radiation that is influenced by fluctuating meteorological conditions. Thus the development of forecast methods capable of determining the level of power generation (e.g., through wind or solar power) in near real-time is needed. Another scenario is the determination of optimal locations for power plants. These aspects are considered in the domain of energy meteorology. For that purpose large data repositories from many heterogeneous sources (e.g., satellites, earth stations, and data archives) are the base for complex computations. The idea is to parallelize these computations in order to obtain significant speed-ups. This paper reports on employing Grid technologies within an ongoing project, which aims to set up a Grid infrastructure among several geographically distributed project partners. An approach to transfer large data sets from many heterogenous data sources and a means of utilizing parallelization are presented. For this purpose we are evaluating various Grid middleware platforms. In this paper we report on our experience with Globus Toolkit 4, Condor, and our first experiments with UNICORE.

Keywords

Data Transfer Message Passing Interface Grid Service Project Partner Grid Infrastructure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Foster, I.: What is the Grid? A Three Point Checklist. Grid Today 1(6) (2002), http://www.gridtoday.com/02/0722/100136.html
  2. 2.
    Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the Grid: Enabling scalable virtual organizations. In: Sakellariou, R., Keane, J.A., Gurd, J.R., Freeman, L. (eds.) Euro-Par 2001. LNCS, vol. 2150, pp. 2–13. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  3. 3.
    MPI: Message Passing Interface. http://www-unix.mcs.anl.gov/mpi/standard.html. Retrieved: 2006-06-11
  4. 4.
    Globus: The Globus Alliance. http://www.globus.org/. Retrieved: 2006-06-11.
  5. 5.
    Globus: Globus Toolkit 4.0 Release Manuals. http://www.globus.org/toolkit/docs/4.0/. Retrieved: 2006-06-11
  6. 6.
    Jacob, B., Brown, M., Fukui, K., Trivedi, N.: Introduction to Grid Computing. http://www.redbooks.ibm.com/redbooks/pdfs/sg246895.pdf. Retrieved: 2006-06-11
  7. 7.
    WSRF: Web Services Resource Framework. http://www.globus.org/wsrf/. Retrieved: 2006-06-11
  8. 8.
    University of Wisconsin: Condor High Throughput Computing. http://www.cs.wisc.edu/condor/. Retrieved: 2006-06-11
  9. 9.
    Mayer, B.: I3RC phase 1 results from the MYSTIC Monte Carlo model. In: I3RC (Intercomparison of 3D Radiation Codes) workshop, Tucson, Arizona (1999)Google Scholar
  10. 10.
    Mayer, B., Kylling, A., Hamann, U.: libRadtran – library for radiative transfer. http://www.libradtran.org. Retrieved: 2006-06-11
  11. 11.
    Geist, A.: PVM: Parallel Virtual Machine: A Users’ Guide and Tutorial for Network Parallel Computing. Scientific and Engineering Computation. MIT Press, Cambridge (1994)Google Scholar
  12. 12.
    EGEE: gLite Lightweight Middleware for Grid Computing. http://glite.web.cern.ch/glite/. Retrieved: 2006-06-11
  13. 13.
    Sun: Sun N1 Grid Engine. http://www.sun.com/software/gridware/. Retrieved: 2006-06-11
  14. 14.
    OGSA-DAI: Open Grid Services Architecture - Data Access and integration. http://www.ogsadai.org.uk/. Retrieved: 2006-06-11
  15. 15.
    Hasselbring, W., Reussner, R.: Toward trustworthy software systems. IEEE Computer 39(4), 91–92 (2006)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Guido Scherp
    • 1
  • Jan Ploski
    • 1
  • Wilhelm Hasselbring
    • 1
    • 2
  1. 1.OFFIS, Escherweg 2, 26121 OldenburgGermany
  2. 2.University of Oldenburg, Software Engineering Group, 26111 OldenburgGermany

Personalised recommendations