Parallel Algorithms for Successive Convolution

Abstract

The development of modern computing architectures with ever-increasing amounts of parallelism has allowed for the solution of previously intractable problems across a variety of scientific disciplines. Despite these advances, multiscale computing problems continue to pose an incredible challenge to modern architectures because they require resolving scales that often vary by orders of magnitude in both space and time. Such complications have led us to consider alternative discretizations for partial differential equations (PDEs) which use expansions involving integral operators to approximate spatial derivatives (Christlieb et al. in J Comput Phys 379:214–236, 2019; Christlieb et al. J Sci Comput 82:52(3):1–29, 2020; Christlieb et al. J Comput Phys 415:1–25, 2020). These constructions use explicit information within the integral terms, but treat boundary data implicitly, which contributes to the overall speed of the method. This approach is provably unconditionally stable for linear problems and stability has been demonstrated experimentally for nonlinear problems. Additionally, it is matrix-free in the sense that it is not necessary to invert linear systems and iteration is not required for nonlinear terms. Moreover, the scheme employs a fast summation algorithm that yields a method with a computational complexity of ${\mathcal {O}}(N)$, where N is the number of mesh points along a coordinate direction. While much work has been done to explore the theory behind these methods, their practicality in large scale computing environments is a largely unexplored topic. In this work, we explore the performance of these methods by developing a domain decomposition algorithm suitable for distributed memory systems along with shared memory algorithms. As a first pass, we derive an artificial Courant–Friedrichs–Lewy condition that enforces a nearest-neighbor (N-N) communication pattern and briefly discuss possible generalizations. We also analyze several approaches for implementing the parallel algorithms by optimizing predominant loop structures and maximizing data reuse. Using a hybrid design that employs MPI and Kokkos (Edwards and Trott in J Parallel Distrib Comput 74:3202–3216, 2014) for the distributed and shared memory components of the algorithms, respectively, we show that our methods are efficient and can sustain an update rate $> 1\times 10^8$ DOF/node/s. We provide results that demonstrate the scalability and versatility of our algorithms using several different PDE test problems, including a nonlinear example, which employs an adaptive time-stepping rule.

Appendices

Example for Linear Advection

Suppose we wish to solve the 1D linear advection equation:

$$\begin{aligned} \partial _{t} u + c \partial _{x} u = 0, \quad (x,t) \in (a,b) \times {\mathbb {R}}^{+}, \end{aligned}$$

(Appendix A.1)

where $c > 0$ is the wave speed and leave the boundary conditions unspecified. The procedure for $c < 0$ is analogous. Discretizing (Appendix A.1) in time with backwards Euler yields a semi-discrete equation of the form

$$\begin{aligned} \frac{u^{n+1}(x) - u^{n}(x)}{\Delta t} + c \partial _{x} u^{n+1}(x) = 0. \end{aligned}$$

If we rearrange this, we obtain a linear equation of the form

$$\begin{aligned} {\mathcal {L}}[u^{n+1}; \alpha ](x) = u^{n}(x), \end{aligned}$$

(Appendix A.2)

where we have used

$$\begin{aligned} \alpha := \frac{1}{c\Delta t},\quad {\mathcal {L}}:= {\mathcal {I}} + \frac{1}{\alpha } \partial _{x}. \end{aligned}$$

By reversing the order in which the discretization is performed, we have created a sequence of BVPs at discrete time levels. If we had discretized equation (Appendix A.1) using the MOL formalism, then ${\mathcal {L}}$ would be an algebraic operator. To solve Eq. (Appendix A.2) for $u^{n+1}$, we analytically invert the operator ${\mathcal {L}}$. Notice that this equation is actually an ODE, which is linear, so the problem can be solved using methods developed for ODEs. If we apply the integrating factor method to the problem, we obtain

$$\begin{aligned} \partial _{x} \left[ e^{\alpha x} u^{n+1}(x) \right] = \alpha e^{\alpha x} u^{n}(x). \end{aligned}$$

To integrate this equation, we use the fact that characteristics move to the right, so integration is performed from a to x. After rearranging the result, we arrive at the update equation

$$\begin{aligned} u^{n+1}(x)&= e^{-\alpha (x-a)} u^{n+1}(a) + \alpha \int _{a}^{x} e^{-\alpha (x - s)} u^{n}(s) \, ds, \\&\equiv e^{-\alpha (x-a)} A^{n+1} + \alpha \int _{a}^{x} e^{-\alpha (x - s)} u^{n}(s) \, ds, \\&\equiv {\mathcal {L}}^{-1}[u^{n}; \alpha ](x). \end{aligned}$$

This update displays the origins of the implicit behavior of the method. While convolutions are performed on data from the previous time step, the boundary terms are taken at time level $n+1$.

Now that we have obtained the update equation, we need to apply the boundary conditions. Clearly, if the problem specifies a Dirchlet boundary condition at $x = a$, then $A^{n+1} = u^{n+1}(a)$. We can compute a variety of boundary conditions using the update equation

$$\begin{aligned} u^{n+1}(x) = e^{-\alpha (x-a)} A^{n+1} + \alpha \int _{a}^{x} e^{-\alpha (x - s)} u^{n}(s) \, ds, \end{aligned}$$

where

$$\begin{aligned} I[u^{n};\alpha ](x) = \alpha \int _{a}^{x} e^{-\alpha (x - s)} u^{n}(s) \, ds. \end{aligned}$$

For example, with periodic boundary conditions, we would need to satisfy

$$\begin{aligned} u^{n+1}(a)&= u^{n+1}(b), \end{aligned}$$

(Appendix A.3)

$$\begin{aligned} \partial _{x}u^{n+1}(a)&= \partial _{x} u^{n+1}(b). \end{aligned}$$

(Appendix A.4)

Applying condition (Appendix A.3), we find that

$$\begin{aligned} A^{n+1} = e^{-\alpha (b-a)} A^{n+1} + \alpha \int _{a}^{b} e^{-\alpha (b - s)} u^{n}(s) \, ds. \end{aligned}$$

Solving this equation for $A^{n+1}$ shows that

$$\begin{aligned} A^{n+1} = \frac{I[u^{n};\alpha ](b)}{1 - \mu }, \end{aligned}$$

with $\mu = e^{-\alpha (b - a)}$. Alternatively, we could have started with (Appendix A.4), which would give an identical solution. While this particular procedure is only applicable to linear problems, this exercise motivates some of the choices made to define operators in the method.

Kokkos Kernels

This section provides listings, which outline the general format of the Kokkos kernels used in this work. Specifically, we provide structures for the tiled/blocked algorithms (Listing 3) in addition to the kernel that executes the fast summation method along a line (Listing 3).

Sixth-Order WENO Quadrature

We provide the various expressions for the coefficients and smoothness indicators used in the reconstruction process for $J_{R}^{(r)}$. Defining $\nu \equiv \alpha \Delta x$, the coefficients for the fixed stencils are given in [2] as follows:

$$\begin{aligned} c_{-3}^{(0)}&= \frac{6 - 6\nu + 2\nu ^{2} - (6-\nu ^{2})e^{-\nu } }{6\nu ^{3}}, \\ c_{-2}^{(0)}&= -\frac{6 - 8\nu + 3\nu ^{2} - (6 - 2\nu -2\nu ^{2})e^{-\nu } }{2\nu ^{3}}, \\ c_{-1}^{(0)}&= \frac{6 - 10\nu + 6\nu ^{2} - (6 - 4\nu -\nu ^{2} + 2\nu ^{2})e^{-\nu } }{2\nu ^{3}}, \\ c_{0}^{(0)}&= -\frac{6 - 12\nu + 11\nu ^{2} - 6\nu ^{3} - (6 - 6\nu + 2\nu ^{2})e^{-\nu } }{6\nu ^{3}}, \\&~\\ c_{-2}^{(1)}&= \frac{6 - \nu ^{2} - (6 + 6\nu + 2\nu ^{2})e^{-\nu } }{6\nu ^{3}}, \\ c_{-1}^{(1)}&= -\frac{6 - 2\nu - 2\nu ^{2} - (6 + 4\nu - \nu ^{2} - 2\nu ^{3})e^{-\nu } }{2\nu ^{3}}, \\ c_{0}^{(1)}&= \frac{6 - 4\nu - \nu ^{2} + 2\nu ^{3} - (6 + 2\nu - 2\nu ^{2} )e^{-\nu } }{2\nu ^{3}}, \\ c_{1}^{(1)}&= -\frac{6 - 6\nu + 2\nu ^{2} - (6-\nu ^{2})e^{-\nu } }{6\nu ^{3}}, \\&~\\ c_{-1}^{(2)}&= \frac{6 + 6\nu + 2\nu ^{2} - (6 + 12\nu + 11\nu ^{2} + 6\nu ^{3} )e^{-\nu } }{6\nu ^{3}}, \\ c_{0}^{(2)}&= -\frac{6 + 4\nu - \nu ^{2} - 2\nu ^{3} - (6 + 10\nu + 6\nu ^{2})e^{-\nu } }{2\nu ^{3}}, \\ c_{1}^{(2)}&= \frac{6 + 2\nu - 2\nu ^{2} - (6 + 8\nu + 3\nu ^{2})e^{-\nu } }{2\nu ^{3}}, \\ c_{2}^{(2)}&= -\frac{6 - \nu ^{2} - (6 + 6\nu + 2\nu ^{2})e^{-\nu } }{6\nu ^{3}}. \end{aligned}$$

The corresponding linear weights are

$$\begin{aligned} d_{0}&= \frac{6 - \nu ^{2} - (6 + 6\nu + 2\nu ^{2})e^{-\nu } }{3\nu ( 2 - \nu - (2 + \nu )e^{-\nu } ) }, \\ d_{2}&= \frac{ 60 - 60\nu + 15\nu ^{2} + 5\nu ^{3} - 3\nu ^{4} - (60 - 15\nu ^{2} + 2\nu ^{4})e^{-\nu } }{ 10\nu ^{2} (6 - \nu ^{2} - (6 + 6\nu + 2\nu ^{2})e^{-\nu } ) }, \\ d_{1}&= 1 - d_{0} - d_{2}. \end{aligned}$$

To obtain the analogous expressions for $J_{L}^{(r)}$, we exploit the “mirror-symmetry" property of WENO reconstructions. That is, one can keep the left side of each of the expressions, then reverse the order of the expressions on the right. Expressions for calculating one particular smoothness indicator, if interested, can be found in [2].