Abstract
The National Computational Infrastructure (NCI) at the Australian National University (ANU) has co-located a priority set of over 10 PetaBytes (PBytes) of national data collections within a HPC research facility. The facility provides an integrated high-performance computational and storage platform, or a High Performance Data (HPD) platform, to serve and analyse the massive amounts of data across the spectrum of environmental collections – in particular from the climate, environmental and geoscientific domains. The data is managed in concert with the government agencies, major academic research communities and collaborating overseas organisations. By co-locating the vast data collections with high performance computing environments and harmonising these large valuable data assets, new opportunities have arisen for Data-Intensive interdisciplinary science at scales and resolutions not hitherto possible.
Chapter PDF
Similar content being viewed by others
Keywords
References
Department of Education: National Collaboration Research Infrastructure Strategy, retrieved from https://education.gov.au/national-collaborative-research-infrastructure-strategy-ncrison (retrieve on November 27, 2014)
Department of Education: Information Sheet Research Data Storage Initiative (RDSI) Australian Government, http://docs.education.gov.au/documents/information-sheet-research-data-storage-initiative-rdsi (modified September 24, 2014)
Hey, T., Tansley, S., Tolle, K.: Jim Grey on eScience: a transformed scientific method. In: Hey, T., Tansley, S., Tolle, K. (eds.) The Fourth Paradigm: Data-Intensive Science Discovery, pp. xvii–xxxi. Microsoft Corporation, USA (2009)
National Computational Infrastructure: National Collections, retrieved from http://nci.org.au/data-collections/data-collections/ , and the GeoNetwork catalogue http://nci.org.au/data-collections/data-collections/and (November 27, 2014)
Taylor, K.E., Stouffer, R.J., Meehl, G.A.: An Overview of CMIP5 and the experiment design. Bull. Amer. Meteor. Soc. 93, 485–498 (2012), doi:10.1175/BAMS-D-11-00094.1
Japan Meteorological Agency: Himarwari-8/9 of the Meteorological Satellite Center (MSC) of the JMA, retrieved from http://www.data.jma.go.jp/mscweb/en/himawari89/index.html (November 27, 2014)
European Space Agency: The Copernicus programme, retrieved from http://www.esa.int/Our_Activities/Observing_the_Earth/Copernicus/Overview4 (November 27, 2014)
Open Geospatial Consortium Web Processing Service, OGC, retrieved from http://www.opengeospatial.org/standards/wps (November 27, 2014)
OpenStack, retrieved from http://www.openstack.org/on (November 27, 2014)
Lustre file system, retrieved from http://wiki.lustre.org/index.php/Main_Page (November 27, 2014)
PuppetLabs Inc., retrieved from http://puppetlabs.com/ (November 27, 2014)
Git, retrieved from http://git-scm.com/ (November 27, 2014)
Docker, retrieved from https://www.docker.com/ (November 27, 2014)
Open Geospatial Consortium: Web Map Service, retrieved from http://www.opengeospatial.org/standards/wms (November 27, 2014)
Open Geospatial Consortium: Web Coverage Service, OGC, retrieved from http://www.opengeospatial.org/standards/wcs (November 27, 2014)
Open Geospatial Consortium: Web Feature Service, OGC, retrieved from http://www.opengeospatial.org/standards/wfs (November 27, 2014)
Open Source Geospatial Foundation:GeoNetwork, retrieved from http://geonetwork-opensource.org/ (modified January 31, 2014)
ElasticSearch, retrieved from http://www.elasticsearch.org/ (November 27, 2014)
THREDDS, retrieved from http://www.unidata.ucar.edu/software/thredds/current/tds/ (November 27, 2014)
OpenDAPInc: Hyrax, retrieved from http://docs.opendap.org/index.php/Hyrax (November 27, 2014)
Open Source Geospatial Foundation: GeoServer, retrieved from http://geoserver.org/ (November 27, 2014)
JSON, retrieved from http://www.json.org/ (November 27, 2014)
Australian Geoscience Data Cube (AGDC), retrieved from https://github.com/GeoscienceAustralia/agdc/wikion (November 27, 2014)
Earth Systems Grid Federation: About the Earth System Grid, retrieved, from https://www.earthsystemgrid.org/about/overview.htm (November 27, 2014)
Oliver, H.: The Cylc Suite Engine, retrieved from http://cylc.github.io/cylc/html/single/cug-html.html (November 27, 2014)
UK Met Office: Rose, retrieved from https://github.com/metomi/rose/ and http://www.metoffice.gov.uk/research/collaboration/rose (November 27, 2014)
Williams, D.N., et al.: UV-CDAT (2014), doi:10.5281/zenodo.12251, http://uvcdat.llnl.gov/
Oceans Data Interoperability Platform (ODIP), retrieved from http://www.odip.org/ (November 27, 2014)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 IFIP International Federation for Information Processing
About this paper
Cite this paper
Evans, B. et al. (2015). The NCI High Performance Computing and High Performance Data Platform to Support the Analysis of Petascale Environmental Data Collections. In: Denzer, R., Argent, R.M., Schimak, G., Hřebíček, J. (eds) Environmental Software Systems. Infrastructures, Services and Applications. ISESS 2015. IFIP Advances in Information and Communication Technology, vol 448. Springer, Cham. https://doi.org/10.1007/978-3-319-15994-2_58
Download citation
DOI: https://doi.org/10.1007/978-3-319-15994-2_58
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15993-5
Online ISBN: 978-3-319-15994-2
eBook Packages: Computer ScienceComputer Science (R0)