Shelter from the Storm: Building a Safe Archive in a Hostile World
The storing of data and configuration files related to scientific experiments is vital if those experiments are to remain reproducible, or if the data is to be shared easily. The prescence of historical (observed) data is also important in order to assist in model evaluation and development. This paper describes the design and implementation process for a data archive, which was required for a coastal modelling project.
The construction of the archive is described in detail, from its design through to deployment and testing. As we will show, the archive has been designed to tolerate failures in its communications with external services, and also to ensure that no information is lost if the archive itself fails, i.e. upon restarting, the archive will still be in exactly the same state.
Unable to display preview. Download preview PDF.
- 1.Abdelnur, A., Hepper, S.: JavaTMPortlet Specification Version 1.0, Java Specification Request 168 (JSR 168), Commuinity Development of Java Technology Specifications (October 2003), Online at, http://jcp.org/en/jsr/detail?id=168
- 2.Allcock, W. (ed.): GridFTP: Protocol Extensions to FTP for the Grid, Global Grid Forum Recommendation Document GFD.20. Online at, http://www.ggf.org/documents
- 5.Ballinger, K., Ehnebuske, D., et al. (eds.): Basic Profile Version 1.1 (Final), The Web Services-Interoperability Organization, WS-I (August 2004), Online at http://www.ws-i.org/Profiles/BasicProfile-1.1-2004-08-24.html
- 8.Mandrichenko, I., Allcock, W., Perelmutov, T. (eds.): GridFTP v2 Protocol Description, Global Grid Forum Recommendation Document GFD.47. Online at, http://www.ggf.org/documents
- 10.Rajasekar, A., Wan, M., Moore, R.: MySRB and SRB - Components of a Data Grid. In: Proceedings of the 11th International Symposium on High Performance Distributed Computing (HPDC-11), Edinburgh, Scotland, pp. 301–310 (July 2002)Google Scholar
- 11.Ris, R.C., Holthuijsen, L.H., Booij, N.: A Spectral Model for Waves in the Near Shore Zone. In: Proceedings of the 24th International Conference on Coastal Engineering, Kobe, Japan, pp. 68–78 (October 1994)Google Scholar
- 12.Waldo, J., Wyant, G., Wollrath, A., Kendall, S.: A Note on Distributed Computing. Technical Report TR-94-29, Sun Microsystems Laboratories, Inc. (November 1994), Online at, http://research.sun.com/techrep/1994/abstract-29.html
- 13.Westerink, J., Luettich, R., Baptista, A., Scheffner, N., Farrar, P.: Tide and Storm Surge Predictions Using Finite Element Model. ASCE Journal of Hydraulic Engineering, 1373–1390 (1992)Google Scholar
- 14.Unidata’s Network Common Data Form (NetCDF), http://my.unidata.ucar.edu/content/software/netcdf/index.html
- 15.The SCOOP Program’s Coastal Model Project Website, http://www1.sura.org/3000/3310_Scoop.html