Abstract
In this paper, a fully implicit finite volume Eulerian scheme and a corresponding scalable parallel solver are developed for some tracer transport problems on the cubed-sphere. To efficiently solve the large sparse linear system at each time step on parallel computers, we introduce a Schwarz preconditioned Krylov subspace method using two discretizations. More precisely speaking, the higher order method is used for the residual calculation and the lower order method is used for the construction of the preconditioner. The matrices from the two discretizations have similar sparsity pattern and eigenvalue distributions, but the matrix from the lower order method is a lot sparser, as a result, excellent scalability results (in total computing time and the number of iterations) are obtained. Even though Schwarz preconditioner is originally designed for elliptic problems, our experiments indicate clearly that the method scales well for this class of purely hyperbolic problems. In addition, we show numerically that the proposed method is highly scalable in terms of both strong and weak scalabilities on a supercomputer with thousands of processors.
Similar content being viewed by others
References
Balay, S., Buschelman, K., Gropp, W.D., Kaushik, D., Knepley, M., McInnes, L.C., Smith, B.F., Zhang, H.: PETSc Users Manual. Argonne National Laboratory (2012)
Brown, P.N., Shumaker, D.E., Woodward, C.S.: Fully implicit solution of large-scale non-equilibrium radiation diffusion with high order time integration. J. Comput. Phys. 204, 760–783 (2005)
Cai, X.-C., Gropp, W.D., Keyes, D.E., Melvin, R.G., Young, D.P.: Parallel Newton-Krylov-Schwarz algorithms for the transonic full potential equation. SIAM J. Sci. Comput. 19, 246–265 (1998)
Cai, X.-C., Sarkis, M.: A restricted additive Schwarz preconditioner for general sparse linear systems. SIAM J. Sci. Comput. 21, 792–797 (1999)
Chen, C., Xiao, F.: Shallow water model on cubed-sphere by multi-moment finite volume method. J. Comput. Phys. 227, 5019–5044 (2008)
Evans, K.J., Knoll, D.A.: Temporal accuracy of phase change convection simulations using the JFNK-SIMPLE algorithm. Int. J. Num. Meth. Fluids. 55, 637–655 (2007)
Erath, C., Lauritzen, P.H., Garcia, J.H., Tufo, H.M.: Integrating a scalable and efficient semi-Lagrangian multi-tracer transport scheme in HOMME. Proc. Comput. Sci. 9, 994–1003 (2012)
Harris, L.M., Lauritzen, P.H., Mittal, R.: A flux-form version of the conservative semi-Lagrangian multi-tracer transport scheme (CSLAM) on the cubed-sphere grid. J. Comput. Phys. 230, 1215–1237 (2011)
Jacobson, M.Z.: Fundamentals of Atmospheric Modeling. Cambridge University Press, New York (1999)
Knoll, D.A., Chacon, L., Margolin, L.G., Mousseau, V.A.: On balanced approximations for time integration of multiple time scale systems. J. Comput. Phys. 185, 583–611 (2003)
Lauritzen, P.H., Nair, R.D., Ullrich, P.A.: A conservative semi-Lagrangian multi-tracer transport scheme (CSLAM) on the cubed-sphere grid. J. Comput. Phys. 229, 1401–1424 (2010)
Lauritzen, P.H., Skamarock, W.C.: Test-case suite for 2D passive tracer transport: a proposal for the NCAR transport workshop. March (2011)
Lauritzen, P.H., Jablonowski, C., Taylor, M., Nair, R.: Numerical Techniques for Global Atmospheric Models. Lecture Notes in Computational Science and Engineering. Springer, Berlin (2011)
Lauritzen, P.H., Ullrich, P.A., Nair, R.D.: Atmospheric transport schemes: desirable properties and a semi-Lagrangian view on finite-volume discretizations. In: Lecture Notes in Computational Science and Engineering (Tutorials), vol. 80, Springer, (2011)
Lauritzen, P.H., Skamarock, W.C., Prather, M.J., Taylor, M.A.: A standard test case suite for two-dimensional linear transport on the sphere. Geosci. Model Dev. Discuss. 5, 189–228 (2012)
Lauritzen, P.H., Thuburn, J.: Evaluating advection/transport schemes using interrelated tracers, scatter plots and numerical mixing diagnostics. Q. J. Roy. Meteor. Soc. 138, 906–918 (2012)
Nair, R.D., Thomas, S.J., Loft, R.D.: A discontinuous Galerkin global shallow water model. Mon. Weather Rev. 133, 876–888 (2005)
Nair, R.D., Lauritzen, P.H.: A class of deformational-flow test cases for linear transport problems on the sphere. J. Comput. Phys. 229, 8868–8887 (2010)
Putman, W.M., Lin, S.-J.: Finite-volume transport on various cubed-sphere grids. J. Comput. Phys. 227, 55–78 (2007)
Rancic, M.R., Purser, J., Mesinger, F.: A global-shallow water model using an expanded spherical cube: Gnomonic versus conformal coordinates. Q. J. Roy. Meteor. Soc. 122, 959–982 (1996)
Ronchi, C., Iacono, R., Paolucci, P.: The cubed sphere: a new method for the solution of partial differential equations in spherical geometry. J. Comput. Phys. 124, 93–114 (1996)
Sadourny, R., Arakawa, A., Mintz, Y.: Integration of the nondivergent barotropic vorticity equation with an icosahedralhexagonal grid for the sphere. Mon. Weather Rev. 96, 351–356 (1968)
Sadourny, R.: Conservative finite-difference approximations of the primitive equations on quasi-uniform spherical grids. Mon. Weather Rev. 100, 211–224 (1972)
Saad, Y.: Iterative Methods for Sparse Linear Systems. SIAM, Philadelphia (2003)
Shadid, J.N., Tuminaro, R.S., Devine, K.D., Hennigan, G.L., Lin, P.T.: Performance of fully coupled domain decomposition preconditioners for finite element transport/reaction simulations. J. Comput. Phys. 205, 24–47 (2005)
Smith, B., Bjørstad, P., Gropp, W.: Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations. Cambridge University Press, Cambridge (1996)
Toselli, A., Widlund, O.: Domain Decomposition Methods-Algorithms and Theory. Springer, Berlin (2005)
Van Albada, G.D., van Leer, B., Roberts, W.W.: A comparative study of computational methods in cosmic gas dynamics. Astron. Astrophys. 108, 95–103 (1982)
White III, J.B., Dongarra, J.J.: High-performance high-resolution semi-Lagrangian tracer transport on a sphere. J. Comput. Phys. 230, 6778–6799 (2011)
Williamson, D.L., Drake, J.B., Hack, J.J., Jakob, R., Swarztrauber, P.N.: A standard test set for numerical approximations to the shallow water equations in spherical geometry. J. Comput. Phys. 102, 211–224 (1992)
Wu, Y., Cai, X.-C., Keyes, D.E.: Additive Schwarz methods for hyperbolic equations. In: Mandel, J., Farhat, C., Cai, X.-C. (eds.) Proceedings of the 10th International Conference on Domain Decomposition Methods, AMS, pp. 513–521 (1998)
Yang, C., Cao, J., Cai, X.-C.: A fully implicit domain decomposition algorithm for shallow water equations on the cubed-sphere. SIAM J. Sci. Comput. 32, 418–438 (2010)
Yang, C., Cai, X.-C.: Parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on cubed-sphere. J. Comput. Phys. 230, 2523–2539 (2011)
Yang, C., Cai, X.-C.: A scalable fully implicit compressible Euler solver for mesoscale nonhydrostatic simulation of atmospheric flows. SIAM J. Sci. Comput. To appear
Yang, H., Cai, X.-C.: Parallel two-grid semismooth Newton-Krylov-Schwarz method for nonlinear complementarity problems. J. Sci. Comput. 47, 258–280 (2011)
Yang, H., Prudencio, E., Cai, X.-C.: Fully implicit Lagrange-Newton-Krylov-Schwarz algorithms for boundary control of unsteady incompressible flows. Int. J. Numer. Meth. Eng. 91, 644–665 (2012)
Zhang, J., Wang, L.L., Rong, Z.: A prolate-element method for nonlinear PDEs on the sphere. J. Sci. Comput. 47, 73–92 (2011)
Acknowledgments
The authors would like to express their appreciations to the anonymous reviewers for the invaluable comments that greatly improved the quality of the manuscript. This work was supported in part by NSF grant CCF-1216314 and DOE grant DE-SC0001774. H. Yang was also supported in part by NSFC grants 91330111, 11201137 and 11272352. C. Yang was also supported in part by NSFC grants 61170075 and 91130023
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yang, H., Yang, C. & Cai, XC. Parallel Domain Decomposition Methods with Mixed Order Discretization for Fully Implicit Solution of Tracer Transport Problems on the Cubed-Sphere. J Sci Comput 61, 258–280 (2014). https://doi.org/10.1007/s10915-014-9828-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10915-014-9828-y