High Level File System and Parallel I/O Optimization of DNS Code
The representation of clouds and convection has an enormous impact on simulation of the climate system. Clouds play key role in behavior of weather elements. They have a huge impact on earth radiation budget system, water and energy cycle and hence controlling the climate. There are lot of biases in current atmospheric general circulation models because of inadequate understanding of micro-physical processes and the hydrodynamics in clouds. For improving the models, small scale simulation such as Direct Numerical Simulation (DNS) is required which solves equations without any assumptions. DNS is very complex and requires huge amount of computational resources. These simulations are memory intensive as well as consuming large space for I/O. Writing huge files at run-time is a challenging task and requires optimization in parallel I/O. This study presents different types of optimization for Lustre file system and advanced vector extension 2. The optimization has improved the code by reducing total simulation time significantly.
KeywordsDNS Parallel I/O optimization Striping NUMA optimization IOBUF optimization
The HPC facilities at IITM Pune is fully funded by Ministry of Earth Science, Government of India.
- 7.NCAR: NCL (NCAR Command Language). https://www.ncl.ucar.edu. Accessed 12 May 2018
- 8.Seagate Technology LLC United States: Lustre: the open source parallel file system. http://lustre.org. Accessed 12 May 2018
- 9.Stommel, H.: Entrainment of air into a cumulus cloud. J. Meteorol. 4(91), 127–129 (1947)Google Scholar
- 10.Northwestern University, Argonne National Laboratory: Parallel netCDF: a parallel I-O library for netCDF file access. https://trac.mcs.anl.gov/projects/parallel-netcdf. Accessed 12 May 2018