Advertisement

SL-AV Model: Numerical Weather Prediction at Extra-Massively Parallel Supercomputer

  • Mikhail TolstykhEmail author
  • Gordey Goyman
  • Rostislav Fadeev
  • Vladimir Shashkin
  • Sergei Lubov
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 965)

Abstract

The SL-AV global atmosphere model is used for operational medium-range and long-range forecasts at Hydrometcentre of Russia. The program complex uses the combination of MPI and OpenMP technologies. Currently, a new version of the model with the horizontal resolution about 10 km is being developed. In 2017, preliminary experiments have shown the scalability of the SL-AV model program complex up to 9000 processor cores with the efficiency of about 45% for grid dimensions of 3024 × 1513 × 51. The profiling analysis for these experiments revealed bottlenecks of the code: non-optimal memory access in OpenMP threads in some parts of the code, time losses in the MPI data exchanges in the dynamical core, and the necessity to replace some numerical algorithms. The review of model code improvements targeting the increase of its parallel efficiency is presented. The new code is tested at the new Cray XC40 supercomputer installed at Roshydromet Main Computer Center.

Keywords

Global atmosphere model Numerical weather prediction Interannual predictability of atmosphere Massively parallel computations Combination of MPI and OpenMP technologies 

Notes

Acknowledgements

This study was carried out at Marchuk Institute of Numerical Mathematics, Russian Academy of Sciences. The study presented in Sects. 2 and 3 was supported with the Russian Science Foundation grant No. 14-27-00126P, the work described in Sect. 4 was supported with the Russian Academy of Sciences Program for Basic Researches No. I.26P.

References

  1. 1.
    WGNE Overview of plans at NWP Centres with Global Forecasting Systems. http://wgne.meteoinfo.ru/nwp-systems-wgne-table/
  2. 2.
    TOP500 Supercomputer sites. https://www.top500.org/
  3. 3.
    Wedi, N.P., et al.: The modelling infrastructure of the integrated forecasting system: recent advances and future challenges. Technical Memorandum 760, ECMWF (2015)Google Scholar
  4. 4.
    Dynamical Core Evaluation Test Report for NOAA’s Next Generation Global Prediction System (NGGPS). https://www.weather.gov/media/sti/nggps/NGGPS%20Dycore%20Phase%202%20Test%20Report%20website.pdf
  5. 5.
    Tolstykh, M.A., et al.: Development of the multiscale version of the SL-AV global atmosphere model. Russ. Meteor. Hydrol. 40, 374–382 (2015).  https://doi.org/10.3103/s1068373915060035CrossRefGoogle Scholar
  6. 6.
    Fadeev, R.Yu., Ushakov, K.V., Kalmykov, V.V., Tolstykh, M.A., Ibrayev, R.A.: Coupled atmosphere–ocean model SLAV–INMIO: implementation and first results. Russ. J. Num. An. Math. Mod. 31, 329–337 (2016).  https://doi.org/10.1515/rnam-2016-0031MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Tolstykh, M., Shashkin, V., Fadeev, R., Goyman, G.: Vorticity-divergence semi-Lagrangian global atmospheric model SL-AV20: dynamical core. Geosci. Model Dev. 10, 1961–1983 (2017).  https://doi.org/10.5194/gmd-10-1961-2017CrossRefGoogle Scholar
  8. 8.
    Geleyn, J.-F., et al.: Atmospheric parameterization schemes in Meteo-France’s ARPEGE N.W.P. model. In: Parameterization of Subgrid-Scale Physical Processes, ECMWF Seminar Proceedings, pp. 385–402. ECMWF, Reading, UK (1994)Google Scholar
  9. 9.
    Gerard, L., Piriou, J.-M., Brožková, R., Geleyn, J.-F., Banciu, D.: Cloud and precipitation parameterization in a Meso-Gamma-Scale operational weather prediction model. Mon. Weather Rev. 137, 3960–3977 (2009).  https://doi.org/10.1175/2009MWR2750CrossRefGoogle Scholar
  10. 10.
    Tarasova, T., Fomin, B.: The use of new parameterizations for gaseous absorption in the CLIRAD-SW solar radiation code for models. J. Atmos. Ocean. Technol. 24, 1157–1162 (2007).  https://doi.org/10.1175/JTECH2023.1CrossRefGoogle Scholar
  11. 11.
    Mlawer, E.J., Taubman, S.J., Brown, P.D.: RRTM, a validated correlated-k model for the longwave. J. Geophys. Res. 102, 16663–16682 (1997).  https://doi.org/10.1029/97jd00237CrossRefGoogle Scholar
  12. 12.
    Volodin, E.M., Lykossov, V.N.: Parametrization of heat and moisture transfer in the soil-vegetation system for use in atmospheric general circulation models: 1. Formulation and simulations based on local observational data. Izvestiya Atmos. Ocean. Phys. 34, 402–416 (1998)Google Scholar
  13. 13.
    Tolstykh, M., Fadeev, R., Goyman, G., Shashkin, V.: Further development of the parallel program complex of SL-AV atmosphere model. In: Voevodin, V., Sobolev, S. (eds.) RuSCDays 2017. CCIS, vol. 793, pp. 290–298. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-71255-0_23CrossRefGoogle Scholar
  14. 14.
    Staniforth, A., Cote, J.: Semi-Lagrangian integration schemes for atmospheric models – a review. Mon. Weather Rev. 119, 2206–2233 (1991)CrossRefGoogle Scholar
  15. 15.
  16. 16.
    White III, J., Dongarra, J.: High-performance high-resolution tracer transport on a sphere. J. Comput. Phys. 230, 6778–6799 (2011).  https://doi.org/10.1016/j.jcp.2011.05.008CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Mikhail Tolstykh
    • 1
    • 2
    • 3
    Email author
  • Gordey Goyman
    • 1
    • 2
  • Rostislav Fadeev
    • 1
    • 2
    • 3
  • Vladimir Shashkin
    • 1
    • 2
    • 3
  • Sergei Lubov
    • 4
  1. 1.Marchuk Institute of Numerical Mathematics Russian Academy of SciencesMoscowRussia
  2. 2.Hydrometcentre of RussiaMoscowRussia
  3. 3.Moscow Institute of Physics and TechnologyDolgorpudnyRussia
  4. 4.Main Computer Center of Federal Service for Hydrometeorology and Environmental MonitoringMoscowRussia

Personalised recommendations