Using Hybrid Grid/Cloud Computing Technologies for Environmental Data Elastic Storage, Processing, and Provisioning

  • Raffaele Montella
  • Ian Foster


High-resolution climate and weather forecast models, and regional and global sensor networks, are producing ever-larger quantities of multidimensional environmental data. To be useful, this data must be stored, managed, and made available to a global community of researchers, policymakers, and others.

The usual approach to addressing these problems is to operate dedicated data storage and distribution facilities. For example, the Earth System Grid (ESG) (Bernholdt et al., 2005) comprises data systems at several US laboratories, each with large quantities of storage and a high-end server configured to support requests from many remote users. Distributed services such as replica and metadata catalogs integrate these different components into a single distributed system.


Virtual Machine Cloud Infrastructure Index Service Common Data Model Virtual Machine Instance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., et al. (2009). Above the clouds: A berkeley view of cloud computing, Electrical Engineering and Computer Sciences University of California, Berkeley, (Technical Report No. UCB/EECS-2009-28)., February 10, 2009.
  2. Ascione, I., Giunta, G., Montella, R., Mariani, P., & Riccio, A. (2006). A grid computing based virtual laboratory for environmental simulations. In W. E. Nagel, W. V. Nagel & W. Lehner (Eds.), Euro-par 2006 parallel processing (pp. 1085–1094). LNCS 4128, Springer, Heidelberg.CrossRefGoogle Scholar
  3. Allcock, B., Bester, J., Bresnahan, J., Chervenak, A. L., Foster, I., Kesselman, C., Meder, S., Nefedova, V., Quesnal, D., & Tuecke, S. (May 2002). Data management and transfer in high performance computational grid environments. Parallel Computing Journal, 28 (5), 749–771.CrossRefGoogle Scholar
  4. Bernholdt, D. et al. (March 2005). The earth system grid: Supporting the next generation of climate modeling research. Proceedings of the IEEE,93(3).Google Scholar
  5. Buyya, R., Yeo, C. S., & Venugopal, S. (2008) Market-oriented cloud computing: Vision, hype, and reality for delivering it services as computing utilities. Proceedings the of 10th IEEE International Conference on High Performance Computing and Communications HPCC ’08, Dalian, China.Google Scholar
  6. Doty, B. E., Kinter III, J. L. (1995) Geophysical data analysis and visualization using GrADS. In E. P. Szuszczewicz & J. H. Bredekamp (Eds.), Visualization techniques in space and atmospheric sciences (pp. 209–219). NASA,Washington, DC.Google Scholar
  7. Foster, I. (July 2002). What is the Grid? A Three Point Checklist.Google Scholar
  8. Foster, I. (2006). Globus toolkit version 4: Software for service-oriented systems. Journal of Computational Science and Technology, 21, 513–520.CrossRefGoogle Scholar
  9. Foster, I., Zhao, Y., Raicu, I., & Lu, S. (2008). Cloud computing and grid computing 360-degree compared. Proceedings of Grid Computing Environments Workshop, GCE ’08, Austin, TX.Google Scholar
  10. Giunta, G., Laccetti, G., & Montella, R. (2008). Five dimension environmental data resource brokering on computational grids and scientific clouds (pp. 81–88), APSCC, IEEE Asia-Pacific Services Computing Conference.Google Scholar
  11. Gallagher, J., Potter, N., Sgouros, T. (2004). DAP Data Model Specification DRAFT, Rev.: 1.68, November 6, 2004.
  12. Gallagher, J., Potter, N., West, P., Garcia, J., & Fox, P. (2006). OPeNDAP’s Server4: Building a High Performance Data Server for the DAP Using Existing Software, AGU Meeting in San Francisco.Google Scholar
  13. Mell, P., Tim, G. (July 2009). The NIST Definition of Cloud Computing, National Institute of Standards and Technology, Version 15. Information Technology Laboratory.Google Scholar
  14. Montella, R., Agrillo, G., & Di Lauro, R. (April 2008). Abstract Instrument Framework: Java Interface for Instrument Abstraction,” (DSA Technical Report. Napoli).Google Scholar
  15. Montella, R., Agrillo, G., Mastrangelo, D., & Menna, M. (June 2008). A globus toolkit 4 based instrument service for environmental data acquisition and distribution. Proceedings of Upgrade Content Workshop HPDC2008. Boston, MA.Google Scholar
  16. Montella, R., Giunta, G., & Riccio, A. (June 2007). Using grid computing based components in on demand environmental data delivery. Proceedings of upgrade content Workshop HPDC2007. Monterey Bay.Google Scholar
  17. Montella, R., & Agrillo, G. (April 2009). GrADSj: A GrADS Java interface (DSA Technical Report, Napoli).Google Scholar
  18. Montella, R., & Agrillo, G. (June 2009). Abstract Execution Framework: Java Interface for Out of the Process Execution, (DSA Technical Report, Napoli).Google Scholar
  19. Rew, R. K., & Davis, G. P. (July 1990). NetCDF: An interface for scientific data access. IEEE Computer Graphics and Applications, 10(4), 76–82.CrossRefGoogle Scholar
  20. Sotomayor, B., Keahey, K., & Foster, I. (June 2008). Combining batch execution and leasing using virtual machines. ACM/IEEE International Symposium on High Performance Distributed Computing 2008 (HPDC 2008), Boston, MA.Google Scholar
  21. Wielgosz, J. & Doty, J. A. B. (2003). The Grads-Dods Server: An Open-Source Tool for Distributed Data Access and Analysis, 19th International Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology.Google Scholar
  22. Wielgosz, J. (2004). Anagram – A modular java framework for high-performance scientific data servers. 20th International Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Department of Applied ScienceUniversity of Napoli ParthenopeNapoliItaly
  2. 2.Argonne National LaboratoryArgonneUSA
  3. 3.The University of ChicagoChicagoUSA

Personalised recommendations