Journal of Scientific Computing

, Volume 58, Issue 2, pp 275–289 | Cite as

A Parallel Domain Decomposition Method for 3D Unsteady Incompressible Flows at High Reynolds Number

  • Rongliang Chen
  • Yuqi Wu
  • Zhengzheng Yan
  • Yubo Zhao
  • Xiao-Chuan CaiEmail author


Numerical simulation of three-dimensional incompressible flows at high Reynolds number using the unsteady Navier–Stokes equations is challenging. In order to obtain accurate simulations, very fine meshes are necessary, and such simulations are increasingly important for modern engineering practices, such as understanding the flow behavior around high speed trains, which is the target application of this research. To avoid the time step size constraint imposed by the CFL number and the fine spacial mesh size, we investigate some fully implicit methods, and focus on how to solve the large nonlinear system of equations at each time step on large scale parallel computers. In most of the existing implicit Navier–Stokes solvers, segregated velocity and pressure treatment is employed. In this paper, we focus on the Newton–Krylov–Schwarz method for solving the monolithic nonlinear system arising from the fully coupled finite element discretization of the Navier–Stokes equations on unstructured meshes. In the subdomain, LU or point-block ILU is used as the local solver. We test the algorithm for some three-dimensional complex unsteady flows, including flows passing a high speed train, on a supercomputer with thousands of processors. Numerical experiments show that the algorithm has superlinear scalability with over three thousand processors for problems with tens of millions of unknowns.


Three-dimensional unsteady incompressible flows Unstructured finite element Parallel computing Fully implicit Domain decomposition 

Mathematics Subject Classification (2000)

76D05 76F65 65M55 65Y05 


  1. 1.
    Alfonsi, G.: Reynolds-averaged Navier-Stokes equations for turbulence modeling. Appl. Mech. Rev. 62, 040802 (2009)CrossRefGoogle Scholar
  2. 2.
    Alfonsi, G.: On direct numerical simulation of turbulent flows. Appl. Mech. Rev. 64, 020802 (2011)CrossRefGoogle Scholar
  3. 3.
    Balay, S., Buschelman, K., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Smith, B.F., Zhang, H.: PETSc Users Manual. Tech. Rep., Argonne National Laboratory (2012)Google Scholar
  4. 4.
    Bayraktar, E., Mierka, O., Turek, S.: Benchmark computations of 3D laminar flow around a cylinder with CFX, OpenFOAM and FeatFlow. Int. J. Comput. Sci. Eng. 7, 253–266 (2012)CrossRefGoogle Scholar
  5. 5.
    Bazilevs, Y., Calo, V.M., Hughes, T.J.R., Zhang, Y.: Isogeometric fluid-structure interaction: theory, algorithms, and computations. Comput. Mech. 43, 3–37 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Cai, X.-C., Gropp, W.D., Keyes, D.E., Melvin, R.G., Young, D.P.: Parallel Newton-Krylov-Schwarz algorithms for the transonic full potential equation. SIAM J. Sci. Comput. 19, 246–265 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Cai, X.-C., Sarkis, M.: A restricted additive Schwarz preconditioner for general sparse linear systems. SIAM J. Sci. Comput. 21, 792–797 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Chan, T.F., van der Vorst, H.A.: Approximate and incomplete factorizations. In: Parallel Numerical Algorithms, ICASE/LaRC Interdisciplinary Series in Science and Engineering IV. Centenary Conference, Keyes, D.E., Sameh, A., Venkatakrishnan, V. (eds.) Dordrecht. Kluwer, 167–202 (1997)Google Scholar
  9. 9.
    Chen, J.H.: Petascale direct numerical simulation of turbulent combustion-fundamental insights towards predictive models. Proc. Combust. Inst. 33, 99–123 (1999)CrossRefGoogle Scholar
  10. 10.
    Eisenstat, S.C., Walker, H.F.: Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comput. 17, 16–32 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Franca, L.P., Frey, S.L.: Stabilized finite element method: II. The incompressible Navier-Stokes equation. Comput. Methods Appl. Mech. Eng. 99, 209–233 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Friedrich, R., Huttl, T.J., Manhart, M., Wagner, C.: Direct numerical simulation of incompressible turbulent flows. Comput. Fluids 30, 555–579 (2001)CrossRefzbMATHGoogle Scholar
  13. 13.
    Guermond, J.-L., Oden, J.T., Prudhomme, S.: Mathematical perspectives on large eddy simulation models for turbulent flows. J. Math. Fluid Mech. 6, 194–248 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Hwang, F.-N., Cai, X.-C.: A parallel nonlinear additive Schwarz preconditioned inexact Newton algorithm for incompressible Navier-Stokes equations. J. Comput. Phys. 204, 666–691 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Hwang, F.-N., Wu, C.-Y., Cai, X.-C.: Numerical simulation of three-dimensional blood flows using domain decomposition method on parallel computer. J. Chin. Soc. Mech. Eng. 31, 199–208 (2010)Google Scholar
  16. 16.
    John, V.: On the efficiency of linearization schemes and coupled multigrid methods in the simulation of a 3D flow around a cylinder. Int. J. Numer. Meth. Fluids 50, 845–862 (2006)CrossRefzbMATHGoogle Scholar
  17. 17.
    Karypis, G.: METIS/ParMETIS webpage. University of Minnesota, (2012)
  18. 18.
    Mahesh, K., Costantinescu, G., Moin, P.: A numerical method for large-eddy simulation in complex geometries. J. Comput. Phys. 197, 215–240 (2004)CrossRefzbMATHGoogle Scholar
  19. 19.
    Moin, P., Mahesh, K.: Direct numerical simulation: a tool in turbulence research. Annu. Rev. Fluid Mech. 30, 539–578 (1998)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Murillo, M., Cai, X.-C.: A fully implicit parallel algorithm for simulating the nonlinear electrical activity of the heart. Numer. Linear Algebra. Appl. 11, 261–277 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Owen, S.J., Shepherd, J.F.: CUBIT project webpage. (2012)
  22. 22.
    Piomelli, U.: Large-eddy simulation: achievements and challenges. Prog. Aeosp. Sci. 35, 335–362 (1999)CrossRefGoogle Scholar
  23. 23.
    Rahimian, A., Lashuk, I., Veerapaneni, S., Chandramowlishwaran, A., Malhotra, D., Moon, L., Sampath, R., Shringarpure, A., Vetter, J., Vuduc, R., Zorin D., Biros, G.: Petascale direct numerical simulation of blood flow on 200k cores and heterogeneous architectures. In: Proceedings ACM/IEEE Supercomputing Conference, (2010)Google Scholar
  24. 24.
    Saad, Y.: Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston (1996)zbMATHGoogle Scholar
  25. 25.
    Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Sagaut, P.: Large Eddy Simulation for Incompressible Flows. Springer, Berlin (2000)Google Scholar
  27. 27.
    Schäfer, M., Turek, S.: Benchmark computations of laminar flow around a cylinder. Notes Numer. Fluid Mech. 52, 547–566 (1996)CrossRefGoogle Scholar
  28. 28.
    Toselli, A., Widlund, O.: Domain Decomposition Methods: Algorithms and Theory. Springer, Berlin (2005)Google Scholar
  29. 29.
    Yokokawa, M., Itakura, K.I., Uno, A., Ishihara, T., Kaneda, Y.: 16.4-Tflops direct numerical simulation of turbulence by a Fourier spectral method on the Earth Simulator. In: Proceedings ACM/IEEE Supercomputing Conference, (2002)Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Rongliang Chen
    • 1
  • Yuqi Wu
    • 2
  • Zhengzheng Yan
    • 1
  • Yubo Zhao
    • 1
  • Xiao-Chuan Cai
    • 3
    Email author
  1. 1.Shenzhen Institutes of Advanced TechnologyChinese Academy of Sciences ShenzhenChina
  2. 2.Department of Applied MathematicsUniversity of WashingtonSeattleUSA
  3. 3.Department of Computer ScienceUniversity of Colorado BoulderBoulderUSA

Personalised recommendations