Machine Learning Moment Closure Models for the Radiative Transfer Equation III: Enforcing Hyperbolicity and Physical Characteristic Speeds

Abstract

This is the third paper in a series in which we develop machine learning (ML) moment closure models for the radiative transfer equation. In our previous work (Huang et al. in J Comput Phys 453:110941, 2022), we proposed an approach to learn the gradient of the unclosed high order moment, which performs much better than learning the moment itself and the conventional $P_N$ closure. However, while the ML moment closure has better accuracy, it is not able to guarantee hyperbolicity and has issues with long time stability. In our second paper (Huang et al., in: Machine learning moment closure models for the radiative transfer equation II: enforcing global hyperbolicity in gradient based closures, 2021. arXiv:2105.14410), we identified a symmetrizer which leads to conditions that enforce that the gradient based ML closure is symmetrizable hyperbolic and stable over long time. The limitation of this approach is that in practice the highest moment can only be related to four, or fewer, lower moments. In this paper, we propose a new method to enforce the hyperbolicity of the ML closure model. Motivated by the observation that the coefficient matrix of the closure system is a lower Hessenberg matrix, we relate its eigenvalues to the roots of an associated polynomial. We design two new neural network architectures based on this relation. The ML closure model resulting from the first neural network is weakly hyperbolic and guarantees the physical characteristic speeds, i.e., the eigenvalues are bounded by the speed of light. The second model is strictly hyperbolic and does not guarantee the boundedness of the eigenvalues. Several benchmark tests including the Gaussian source problem and the two-material problem show the good accuracy, stability and generalizability of our hyperbolic ML closure model.

Appendix A: Collections of Proofs

In this appendix, we collect some lemma and proofs. We start with a lemma which characterize the eigenspace of unreduced lower Hessenberg matrix.

Lemma A.1

For an unreduced lower Hessenberg matrix $H=(h_{ij})_{n\times n}$, the geometric multiplicity of any eigenvalue $\lambda$ is 1 and the corresponding eigenvector is $(q_0(\lambda ), q_1(\lambda ), \ldots , q_{n-1}(\lambda ))^T$. Here $\{q_i\}_{0\le i\le n-1}$ is the associated polynomial sequence defined in (2.1).

Proof

By Definition 2.1, we have that $h_{ij}=0$ for $j>i+1$ and $h_{i,i+1}\ne 0$ for $i=1,\ldots ,n-1$. Let $r=(r_1,r_2,\ldots ,r_n)$ be an eigenvector associated with $\lambda$. We write $Ar = \lambda r$ as an equivalent component-wise formulation:

$$\begin{aligned} \sum _{j=1}^i h_{ij} r_j + h_{i,i+1} r_{i+1} = \lambda r_i, \quad i = 1,\ldots ,n-1, \end{aligned}$$

(A.1)

and

$$\begin{aligned} \sum _{j=1}^n h_{nj} r_j = \lambda r_n. \end{aligned}$$

(A.2)

Here we used the fact that $h_{ij}=0$ for $j>i+1$. Since $h_{i,i+1}\ne 0$ for $i=1,\ldots ,n-1$, (A.1) is equivalent to

$$\begin{aligned} r_{i+1} = \frac{1}{h_{i,i+1}} \left( \lambda r_i - \sum _{j=1}^i h_{ij} r_j\right) , \quad i = 1,\ldots ,n-1 \end{aligned}$$

(A.3)

From (A.3), we deduce that $r_1\ne 0$, otherwise $r_2=\cdots =r_n=0$. Moreover, $r_i$ for $i = 2,\ldots ,n$ are uniquely determined by $r_1$. Therefore, the geometric multiplicity of $\lambda$ is 1. Moreover, without loss of generality, we take $r_1=1$. In this case, r is exactly the same with $(q_0(\lambda ),q_1(\lambda ),\ldots ,q_{n-1}(\lambda ))^T$. Here $\{q_i\}_{0\le i\le n-1}$ is the associated polynomial sequence defined in (2.1). $\square$

Lemma A.2

Let $H = (h_{ij})_{n\times n}$ be an unreduced lower Hessenberg matrix and $\{q_i\}_{0\le i\le n}$ is the associated polynomial sequence with H. If $\lambda$ is an eigenvalue of H, then $\lambda$ is a root of $q_n$.

Proof

From Lemma A.1, we have the geometric multiplicity of $\lambda$ is 1 and the corresponding eigenvector $\varvec{q}_{n-1}(\lambda ) = (q_0(\lambda ),q_1(\lambda ),\ldots ,q_{n-1}(\lambda ))^T$, i.e. $H \varvec{q}_{n-1}(\lambda ) = \lambda \varvec{q}_{n-1}(\lambda )$. Plugging $\lambda$ into (2.2), we immediately have $q_n(\lambda )=0$, i.e., $\lambda$ is a root of $q_n$. $\square$

1.1 A.1: Proof of Theorem 2.4

Proof

We start by proving that condition 1 and condition 2 are equivalent. First, it is easy to see that condition 2 implies condition 1. We only need to prove that condition 1 implies condition 2. Since A is real diagonalizable, all the eigenvalues of A are real. Moreover, for any eigenvalue of A, the geometric multiplicity is equal to its algebraic multiplicity. By Lemma A.1, the geometric multiplicity of any eigenvalue of an unreduced lower Hessenberg matrix is 1. Therefore, any eigenvalue of A has algebraic multiplicity of 1, i.e. all the eigenvalues of A are distinct.

Next, we prove that the equivalence of condition 2 and condition 3. It is easy to see that, condition 3 implies condition 2 from Theorem 2.3, and condition 2 implies condition 3 from Lemma A.2. This completes the proof. $\square$

1.2 A.2: Proof of Lemma 3.3

Proof

We start from the definition of Legendre polynomials by the generating function:

$$\begin{aligned} \frac{1}{\sqrt{1-2tx+t^2}} = \sum _{n=0}^\infty P_n(x) t^n. \end{aligned}$$

(A.4)

Introduce the variable s such that

$$\begin{aligned} 1 - ts = \sqrt{1-2tx + t^2}, \end{aligned}$$

(A.5)

which is equivalent to

$$\begin{aligned} x = \frac{1+t^2 - (1-ts)^2}{2t} = s + \frac{t}{2}(1-s^2). \end{aligned}$$

(A.6)

Therefore, we have

$$\begin{aligned}{} & {} \sum _{n=0}^\infty t^n \int _{-1}^1 x^m P_n(x) dx {\mathop {=}\limits ^{\text {(A.4)}}} \int _{-1}^1 \frac{x^m dx}{\sqrt{1-2tx+t^2}}\nonumber \\{} & {} \quad {\mathop {=}\limits ^{\text {(A.6)}}} \int _{-1}^1 \frac{x^m (1-ts) ds}{\sqrt{1-2tx+t^2}} {\mathop {=}\limits ^{\text {(A.5)- (A.6)}}} \int _{-1}^1 \left( s + \frac{t}{2}(1-s^2)\right) ^m ds. \end{aligned}$$

(A.7)

Define

$$\begin{aligned} a_{m,n}:= \int _{-1}^1 x^m P_n(x) dx. \end{aligned}$$

(A.8)

By comparing the coefficients of $t^n$ on both sides of (A.7), we find that $a_{m,n} = 0$ if $n>m$ or m, n has different parity. For $n = m - 2k$ for some integer $k\ge 0$, we have

$$\begin{aligned} a_{m,m-2k} = 2^{2k-m}\left( {\begin{array}{c}m\\ 2k\end{array}}\right) \int _{-1}^1 s^{2k} (1-s^2)^{m-2k} ds = 2^{2k-m}\left( {\begin{array}{c}m\\ 2k\end{array}}\right) \int _{0}^1 2 s^{2k} (1-s^2)^{m-2k} ds\nonumber \\ \end{aligned}$$

(A.9)

By introducing the variable $\tau = s^2$ or equivalently $s = \tau ^{\frac{1}{2}}$, we have

$$\begin{aligned} \begin{aligned} a_{m,m-2k}&= 2^{2k-m}\left( {\begin{array}{c}m\\ 2k\end{array}}\right) \int _{0}^1 2 s^{2k} (1-s^2)^{m-2k} ds \\&= 2^{2k-m}\left( {\begin{array}{c}m\\ 2k\end{array}}\right) \int _{0}^1 2 \tau ^{k} (1-\tau )^{m-2k} \frac{1}{2}\tau ^{-\frac{1}{2}} d\tau \\&= 2^{2k-m}\left( {\begin{array}{c}m\\ 2k\end{array}}\right) \int _{0}^1 \tau ^{k-\frac{1}{2}} (1-\tau )^{m-2k} d\tau \\&= 2^{2k-m}\left( {\begin{array}{c}m\\ 2k\end{array}}\right) \frac{\Gamma (k+\frac{1}{2})\Gamma (m-2k+1)}{\Gamma (m-k+\frac{3}{2})} \\&= \frac{m!}{2^{k-1}k!(2m-2k+1)!!} \end{aligned} \end{aligned}$$

(A.10)

where in the fourth equality we used the relation between the gamma function and the beta function:

$$\begin{aligned} B(x,y):= \int _0^1 t^{x-1} (1-t)^{y-1} dt = \frac{\Gamma (x)\Gamma (y)}{\Gamma (x+y)}, \end{aligned}$$

(A.11)

and in the last equality we used the properties of the gamma function: for any integer $n\ge 0$

$$\begin{aligned} \Gamma (n) = (n-1)!, \quad \Gamma (n+\frac{1}{2}) = \frac{(2n-1)!!}{2^n}\sqrt{\pi }. \end{aligned}$$

(A.12)

Lastly, using the orthogonality relation $\int _{-1}^1 P_m(x)P_n(x) = \frac{2}{2m+1}\delta _{m,n}$, we have for any integer $m\ge 0$,

$$\begin{aligned} x^m = \sum _{k=0}^{\lfloor m/2\rfloor } \left( \frac{2m-4k+1}{2}\right) a_{m,m-2k} P_{m-2k}(x) = \sum _{k=0}^{\lfloor m/2\rfloor }\frac{m!(2m-4k+1)}{2^k k!(2m-2k+1)!!}P_{m-2k}(x)\nonumber \\ \end{aligned}$$

(A.13)

This completes the proof. $\square$