Using Hybrid Grid/Cloud Computing Technologies for Environmental Data Elastic Storage, Processing, and Provisioning
High-resolution climate and weather forecast models, and regional and global sensor networks, are producing ever-larger quantities of multidimensional environmental data. To be useful, this data must be stored, managed, and made available to a global community of researchers, policymakers, and others.
The usual approach to addressing these problems is to operate dedicated data storage and distribution facilities. For example, the Earth System Grid (ESG) (Bernholdt et al., 2005) comprises data systems at several US laboratories, each with large quantities of storage and a high-end server configured to support requests from many remote users. Distributed services such as replica and metadata catalogs integrate these different components into a single distributed system.
KeywordsVirtual Machine Cloud Infrastructure Index Service Common Data Model Virtual Machine Instance
- Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., et al. (2009). Above the clouds: A berkeley view of cloud computing, Electrical Engineering and Computer Sciences University of California, Berkeley, (Technical Report No. UCB/EECS-2009-28). http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.html, February 10, 2009.
- Bernholdt, D. et al. (March 2005). The earth system grid: Supporting the next generation of climate modeling research. Proceedings of the IEEE,93(3).Google Scholar
- Buyya, R., Yeo, C. S., & Venugopal, S. (2008) Market-oriented cloud computing: Vision, hype, and reality for delivering it services as computing utilities. Proceedings the of 10th IEEE International Conference on High Performance Computing and Communications HPCC ’08, Dalian, China.Google Scholar
- Doty, B. E., Kinter III, J. L. (1995) Geophysical data analysis and visualization using GrADS. In E. P. Szuszczewicz & J. H. Bredekamp (Eds.), Visualization techniques in space and atmospheric sciences (pp. 209–219). NASA,Washington, DC.Google Scholar
- Foster, I. (July 2002). What is the Grid? A Three Point Checklist.Google Scholar
- Foster, I., Zhao, Y., Raicu, I., & Lu, S. (2008). Cloud computing and grid computing 360-degree compared. Proceedings of Grid Computing Environments Workshop, GCE ’08, Austin, TX.Google Scholar
- Giunta, G., Laccetti, G., & Montella, R. (2008). Five dimension environmental data resource brokering on computational grids and scientific clouds (pp. 81–88), APSCC, IEEE Asia-Pacific Services Computing Conference.Google Scholar
- Gallagher, J., Potter, N., Sgouros, T. (2004). DAP Data Model Specification DRAFT, Rev.: 1.68, http://www.opendap.org. November 6, 2004.
- Gallagher, J., Potter, N., West, P., Garcia, J., & Fox, P. (2006). OPeNDAP’s Server4: Building a High Performance Data Server for the DAP Using Existing Software, AGU Meeting in San Francisco.Google Scholar
- Mell, P., Tim, G. (July 2009). The NIST Definition of Cloud Computing, National Institute of Standards and Technology, Version 15. Information Technology Laboratory.Google Scholar
- Montella, R., Agrillo, G., & Di Lauro, R. (April 2008). Abstract Instrument Framework: Java Interface for Instrument Abstraction,” (DSA Technical Report. Napoli).Google Scholar
- Montella, R., Agrillo, G., Mastrangelo, D., & Menna, M. (June 2008). A globus toolkit 4 based instrument service for environmental data acquisition and distribution. Proceedings of Upgrade Content Workshop HPDC2008. Boston, MA.Google Scholar
- Montella, R., Giunta, G., & Riccio, A. (June 2007). Using grid computing based components in on demand environmental data delivery. Proceedings of upgrade content Workshop HPDC2007. Monterey Bay.Google Scholar
- Montella, R., & Agrillo, G. (April 2009). GrADSj: A GrADS Java interface (DSA Technical Report, Napoli).Google Scholar
- Montella, R., & Agrillo, G. (June 2009). Abstract Execution Framework: Java Interface for Out of the Process Execution, (DSA Technical Report, Napoli).Google Scholar
- Sotomayor, B., Keahey, K., & Foster, I. (June 2008). Combining batch execution and leasing using virtual machines. ACM/IEEE International Symposium on High Performance Distributed Computing 2008 (HPDC 2008), Boston, MA.Google Scholar
- Wielgosz, J. & Doty, J. A. B. (2003). The Grads-Dods Server: An Open-Source Tool for Distributed Data Access and Analysis, 19th International Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology.Google Scholar
- Wielgosz, J. (2004). Anagram – A modular java framework for high-performance scientific data servers. 20th International Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology.Google Scholar