Computational Mathematics and Mathematical Physics

, Volume 58, Issue 11, pp 1865–1886 | Cite as

FlowModellium Software Package for Calculating High-Speed Flows of Compressible Fluid

  • M. N. Petrov
  • A. A. Tambova
  • V. A. Titarev
  • S. V. Utyuzhnikov
  • A. V. Chikitkin


The development of the software package FlowModellium designed for simulating high-speed flows of continuum medium taking into account nonequilibrium chemical reactions is described. The numerical method and the two-level parallel algorithm used in the package are presented. Examples of computations are discussed.


supersonic flows computational fluid dynamics nonequilibrium chemistry unstructured mesh TVD scheme supercomputer computations. 



This work was supported by the Russian Science Foundation, project no. 18-19-00098.

The research was carried out using the equipment of the shared research facilities of HPC computing resources at Lomonosov Moscow State University [51, 52], resources of the Joint Supercomputer Center (JSCC) of the Russian Academy of Sciences, and of the Peter the Great St. Petersburg Polytechnic University.

We are grateful to D.A. Zabarko, D.A. Sivkov, M.P. Shuvalov, A.N. Krylov, A.V. Beloshitskii, and I.V. Voronich for useful discussions. The work was accomplished using the resources of the Joint Supercomputer Center (JSCC) of the Russian Academy of Sciences, Peter the Great St. Petersburg Polytechnic University, and Lomonosov Moscow State University [51, 52].


  1. 1.
    A. S. Kozelkov, Yu. N. Deryugin, D. K. Zelenskii, V. A. Glazunov, A. A. Golubev, O. V. Denisova, S. V. Lashkin, R. N. Zhuchkov, N. V. Tarasova, and M. A. Sizova, “Multipurpose software package LOGOS for solving fluid dynamics and heat and mass transfer problems on supercomputers: Basic technologies and algorithms,” Proc. of the XII Int. Seminar on Super Computations and Mathematical Modeling, Sarov, Russia, 2010, pp. 215–230.Google Scholar
  2. 2.
    I. V. Abalakin, P. A. Bakhvalov, A. V. Gorobets, A. P. Duben’, and T. K. Kozubskaya, “Parallel software package NOISETTE for large-scale computations in fluid dynamics and aeroacoustics,” Vychicl. Metody Proram. 13 (3), 110–125 (2012).Google Scholar
  3. 3.
    G. A. Faranosov, V. M. Goloviznin, S. A. Karabasov, V. G. Kondakov, V. F. Kopiev, and M. A. Zaitsev, “Cabaret method on unstructured hexahedral grids for jet noise computation,” Comput. Fluids. 88, 165–179 (2013).CrossRefGoogle Scholar
  4. 4.
    A. V. Gorobets, “Parallel Technology for Numerical Modeling of Fluid Dynamics Problems by High-Accuracy Algorithms,” Comput. Math. Math. Phys. 55, 638–649 (2015).MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    A. B. Gorshkov, “Parallelization algorithm for implicit method computation of hypersonic nonequilibrium gas flow past a body, based on Navier–Stokes equations,” Math. Models Comput. Simul. 2, 252–260 (2010).CrossRefGoogle Scholar
  6. 6.
    A. L. Zheleznyakova and S. T. Surzhikov, “Computation of hypersonic flow around bodies with complex shape on unstructured tetrahedral meshes using the AUSM scheme,” Teplomassoobmen Fiz. Gidrodinam. 52 (2), 283–293 (2012).Google Scholar
  7. 7.
    A. V. Novikov, A. V. Fedorov, and I. V. Egorov, “Numerical studies of 3D instabilities propagating in supersonic compression-corner flow,” 8th European Symp. on Aerothermodynamics for Space Vehicles, Lisbon, 2015.Google Scholar
  8. 8.
    V. A. Garanzha, L. N. Kudryavtseva, and S. V. Utyuzhnikov, “Variational method for untangling and optimization of spatial meshes,” J. Comp. Appl. Math. 269, 24–41 (2014).MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    V. A. Titarev and S. V. Utyuzhnikov, “Software package for calculating hypersonic air flows,” Certificate of State Registration of computer programs 2013619670, 2013.Google Scholar
  10. 10.
    S. A. Vasil’evskii, L. G. Efimova, A. F. Kolesnikov, I. A. Sokolova, and G. A. Tirskii, “Highly accurate computation of transport coefficients in a multicomponent plasma. The effect of element separation in chemically and ionization equilibrium plasma,” Rep. 2427 of the Institute of Mechanics, Russian Academy of Sciences, 1980.Google Scholar
  11. 11.
    V. I. Sakharov, “Numerical simulation of thermally and chemically nonequilibrium flows and heat transfer in underexpanded induction plasmatron Jets,” Fluid. Dyn. 42, 1007–10016 (2007).CrossRefzbMATHGoogle Scholar
  12. 12.
    V. I. Sakharov, “Simulation of nonequilibrium flows of viscous gas in induction plasmatrons and around bodies,” Doctoral (Phys. Math.) Dissertation, Moscow State Univ., Moscow, 2011.Google Scholar
  13. 13.
    L. V. Gurvich, I. V. Veits, V. A. Medvedev, et al., Thermodynamic Properties of Individual Materials (Nauka, Moscow, 1979) [in Russian].Google Scholar
  14. 14.
    R. C. Reid, J. M. Prausnitz, and T. K. Sherwood, The Properties of Gases and Liquids (McGraw-Hill, New York, 1977).Google Scholar
  15. 15.
    P. R. Spalart and S. R. Allmaras, “A one-equation turbulence model for aerodynamic flows,” AIAA Paper 92-0439, 1992.Google Scholar
  16. 16.
    J. E. Bardina, P. G. Huang, and T. J. Coakley, “Turbulence modeling validation, testing, and development,” AIAA Paper 92-0439, 1997.Google Scholar
  17. 17.
    P. G. Huang, J. E. Bardina, and T. J. Coakley, “Turbulence modeling validation, testing, and development,” NASA Techn. Rep., 1997.Google Scholar
  18. 18.
    F. Menter, “Zonal two equation kw turbulence models for aerodynamic flows,” 23rd Fluid Dynamics, Plasmadynamics, and Lasers Conf., 1993, p. 2906.Google Scholar
  19. 19.
    M. L. Shur, M. K. Strelets, A. K. Travin, and P. R. Spalart, “Turbulence modeling in rotating and curved channels: Assessing the Spalart–Shur Correction,” AIAA J. 38, 784–792 (2000).CrossRefGoogle Scholar
  20. 20.
    J. R. Edwards and S. Chandra, “Comparison of eddy viscosity–transport turbulence models for three-dimensional, shock-separated flow fields,” AIAA J. 34, 756–763 (1996).CrossRefGoogle Scholar
  21. 21.
    T. Rung, U. Bunge, M. Schatz, and F. Thiele, “Restatement of the Spalart–Allmaras eddy-viscosity model in strain-adaptive formulation,” AIAA J. 41, 1396–1399 (2003).CrossRefGoogle Scholar
  22. 22.
    P. R. Spalart, “Trends in turbulence treatments,” AIAA 2000-2306, 2000.Google Scholar
  23. 23.
    M. Mani, D. A. Babcock, C. M. Winkler, and P. R. Spalart, “Predictions of a supersonic turbulent flow in a square duct,” AIAA Paper 2013-0860, 2013.Google Scholar
  24. 24.
    M. Dumbser, M. Käser, V. A. Titarev, and E. F. Toro, “Quadrature-free non-oscillatory finite volume schemes on unstructured meshes for nonlinear hyperbolic systems,” J. Comput. Phys. 226, 204–243 (2007).MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    P. Tsoutsanis, V. A. Titarev, and D. Drikakis, “WENO schemes on arbitrary mixed-element unstructured meshes in three space dimensions,” J. Comput. Phys. 230, 1585 – 1601 (2010).MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    V. A. Titarev, “Efficient deterministic modelling of three-dimensional rarefied gas flows,” Commun. Comput. Phys. 12 (1), 161–192 (2012).MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    V. A. Titarev, M. Dumbser, and S. V. Utyuzhnikov, “Construction and comparison of parallel implicit kinetic solvers in three spatial dimensions,” J. Comput. Phys. 256, 17–33 (2014).MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    A. G. Kulikovskii, N. V. Pogorelov, and A. Yu. Semenov, Mathematical Issues of the Numerical Solution of Hyperbolic Systems of Equations (Fizmatlit, Moscow, 2001) [in Russian].zbMATHGoogle Scholar
  29. 29.
    E. F. Toro, Riemann Solvers and Numerical Methods for Fluid Dynamics, 3rd ed. (Springer, 2009).CrossRefzbMATHGoogle Scholar
  30. 30.
    A. Harten, P. D. Lax, and B. van Leer, “On upstream differencing and Godunov-type schemes for hyperbolic conservation laws,” SIAM Rev. 25 (1), 35–61 (1983).MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    E. F. Toro, M. Spruce, and W. Speares, “Restoration of the contact surface in the Harten-Lax-van Leer Riemann solver,” J. Shock Waves 4, 25–34 (1994).CrossRefzbMATHGoogle Scholar
  32. 32.
    M. Dumbser and E. F. Toro, “On universal Osher-type schemes for general nonlinear hyperbolic conservation laws,” Commun. Comput. Phys. 10, 635–671 (2011).MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    P. Batten, M. A. Leschziner, and U. C. Goldberg, “Average-state Jacobians and implicit methods for compressible viscous and turbulent Flows, " J. Comput. Phys. 137, 38–78 (1997).MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    V. V. Vlasenko, E. V. Kazhan, E. S. Matyash, S. V. Mikhailov, and A. I. Troshin, “Numerical implementation of an implicit scheme and various turbulence models in the computational module ZEUS,” Trudy Tsentr. Aerogidrodinamich. Inst. 2735, 5–49 (2015).Google Scholar
  35. 35.
    M. Dumbser, J.-M. Moschetta, and J. Gressier, “A matrix stability analysis of the carbuncle phenomenon,” J. Comput. Phys. 192, 647–670 (2004).CrossRefzbMATHGoogle Scholar
  36. 36.
    I. Yu. Tagirova and A. V. Rodionov, “Application of artificial viscosity for suppressing the carbuncle phenomenon in Godunov-type schemes,” Math. Models Comput. Simul. 8, 249–262 (2015).CrossRefzbMATHGoogle Scholar
  37. 37.
    A. V. Rodionov, “Artificial viscosity in Godunov-type schemes to cure the carbuncle phenomenon,” J. Comp. Phys. 345, 308–329 (2017).MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    C. R. Mitchell, “Improved reconstruction schemes for the Navier-Stokes equations on unstructured meshes,” AIAA-94-0642, 1994.Google Scholar
  39. 39.
    Neal T. Frink, “Assessment of an unstructured-grid method for predicting 3-D turbulent viscous flows,” AIAA-96-0292, 1996.Google Scholar
  40. 40.
    S. Yoon and A. Jameson, “Lower-upper symmetric-Gauss-Seidel method for the Euler and Navier–Stokes equations,” AIAA J. 26, 1025–1026 (1998).CrossRefGoogle Scholar
  41. 41.
    I. S. Men’shov and Y. Nakamura, “An implicit advection upwind splitting scheme for hypersonic air flows in thermochemical nonequilibrium,” Collection of Technical Papers of the 6th Int. Symp. on CFD, Lake Tahoe, Nevada, 1995, Vol. 2, p. 815.Google Scholar
  42. 42.
    I. S. Men’shov and Y. Nakamura, “On implicit Godunov’s method with exactly linearized numerical flux,” Comput. Fluids 29, 595–616 (2000).CrossRefzbMATHGoogle Scholar
  43. 43.
    D. Sharov, H. Luo, J. D. Baum, and R. Löhner, “Implementation of unstructured grid GMRES+LU-SGS method on shared-memory, cache-based parallel computers,” 38th Aerospace Sciences Meeting and Exhibit, Reno, Nevada, 2000, AIAA-2000-927, pp. 10–13.Google Scholar
  44. 44.
    M. J. Chorley and D. W. Walker, “Performance analysis of a hybrid MPI/OpenMP application on multi-core clusters,” J. Comput. Sci. 47, 168–174 (2010).CrossRefGoogle Scholar
  45. 45.
    A. V. Gorobets, A. O. Zheleznyakov, S. A. Sukov, P. B. Bogdanov, and B. N. Chetverushkin, “Extension of the two-level MPI + OpenMP parallelization using OpenCL for fluid dynamics computations on heterogeneous systems,” Vest. Yuzhno-Uralsk. Gos. Univ., Ser. Mat. Model. Progr., No. 9, 76–86 (2011).Google Scholar
  46. 46.
    A. V. Gorobets, “Parallel algorithm of the NOISEtte code for CFD and CAA simulations,” Lobachevskii J. Math. 39, 524 – 532 (2018).MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    G. Karypis and V. Kumar, “Multilevel k-way partitioning scheme for irregular graphs,” J. Parallel Distrib. Comput. 48, 96–129 (1998).CrossRefzbMATHGoogle Scholar
  48. 48.
    I. E. Kaporin and O. Yu. Milyukova, “The massively parallel preconditioned conjugate gradient method for the numerical solution of linear algebraic equations,” in Collection of Papers of the Department of Applied Optimization of the Dorodnicyn Computing Center, Russian Academy of Sciences, Ed. by V. G. Zhadan (Vychisl. Tsentr, Ross. Akad. Nauk, Moscow, 2011), pp. 132–157.Google Scholar
  49. 49.
    A. M. Wissink, A. S. Lyrintzis, and R. C. Strawn, “Parallelization of a three-dimensional flow solver for Euler rotorcraft aerodynamics predictions,” AIAA J. 34, 2276–2283 (1996).CrossRefzbMATHGoogle Scholar
  50. 50.
    M. N. Petrov, V. A. Titarev, S. V. Utyuzhnikov, and A. V. Chikitkin, “A multithreaded OpenMP implementation of the LU-SGS method using the multilevel decomposition of the unstructured computational mesh,” Comput. Math. Math. Phys. 57, 1856–1865 (2017).MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Vl. V. Voevodin, S. A. Zhumatii, S. I. Sobolev, A. S. Antonov, P. A. Bryzgalov, D. A. Nikitenko, K. S. Stefanov, and Vad. V. Voevodin, “Practical usage of the Lomonosov supercomputer,” Otkrytye Sist., No. 7, 36–39 (2012).Google Scholar
  52. 52.
    V. Sadovnichy, A. Tikhonravov, Vl. Voevodin, and V. Opanasenko, “Lomonosov: Supercomputing at Moscow State University,” in Contemporary High Performance Computing: From Petascale toward Exascale (Chapman & Hall/CRC Computational Science) (CRC, Boca Raton, USA, 2013), pp. 283–307.Google Scholar
  53. 53.
    A. Semin, E. Druzhinin, V. Mironov, A. Shmelev, and A. Moskovsky, “The performance characterization of the RSC petastream module,” Proc. of the 29th Int. Conf., ISC, Leipzig, 2014; Lect. Notes Comput. Sci. 8488, 420–429 (2014).Google Scholar
  54. 54.
    A. N. Lyubimov and V. V. Rusanov, Gas Flows around Blunt Bodies. Part I: Method of Computation and Analysis of Flows. Part II: Tables of Gasdynamic Functions (Nauka, Moscow, 1970) [in Russian].Google Scholar
  55. 55.
    S. Swaminathan, M. D. Kim, and C. H. Lewis, “Nonequilibrium viscous shock-layer flows over blunt sphere-cones at angle of attack,” J. Spacecraft Rockets 20, 331–338 (1983).CrossRefGoogle Scholar
  56. 56.
    Kelly R. Laflin, Steven M. Klausmeyer, Tom Zickuhr, John C. Vassberg, Richard A. Wahls, Joseph H. Morrison, Olaf P. Brodersen, Mark E. Rakowitz, Edward N. Tinoco, and Jean-Luc Godard, Data Summary from Second AIAA Computational Fluid Dynamics Drag Prediction Workshop; J. Aircraft 42, 1165–1178 (2005).CrossRefGoogle Scholar
  57. 57.
    S. M. Bosnyakov, V. V. Vlasenko, M. F. Engulatova, E. V. Kazhan, S. V. Matyash, and A. I. Troshin, “Industrial solvers in the software package EWT-TsAGI and their verification on a series of standard benchmarks,” Trudy Tsentr. Aerogidrodinamich. Inst. 2735, 3–91 (2014).Google Scholar
  58. 58.
    B. R. Hollis, T. J. Horvath, K. T. Berger, R. P. Lillard, B. S. Kirk, J. J. Coblish, and J. D. Norris, “Experimental investigation of project Orion crew exploration vehicle aeroheating in aedc tunnel 9,” NASA/TP-2008-215547, 2008, p. 158.Google Scholar
  59. 59.
    N. P. Adamov, A. M. Kharitonov, E. A. Chasovnikov, A. A. Dyad’kin, M. I. Kazakov, A. N. Krylov, and A. Yu. Skorovarov, “Aerodynamic characteristics of reentry vehicles at supersonic velocities,” Thermophys. Aeromech. 22, 557–565 (2015).CrossRefGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2018

Authors and Affiliations

  1. 1.Moscow Institute of Physics and TechnologyDolgoprudnyiRussia
  2. 2.Federal Research Center “Computer Science and Control,” Russian Academy of SciencesMoscowRussia
  3. 3.University of ManchesterManchesterUK

Personalised recommendations