Extreme Learning Machine with Kernels for Solving Elliptic Partial Differential Equations

Abstract

Finding solutions for partial differential equations based on machine learning methods is an ongoing challenge in applied mathematics. Although machine learning methods have been successfully applied to solve partial differential equations, their practical applications are limited by a large number of variables. In this study, after a strict theoretical derivation, a new method is proposed for solving standard elliptic partial differential equations based on an extreme learning machine with kernels. The parameters of the proposed method (i.e., regularization coefficients and kernel parameters) were obtained by a grid search approach. Three numerical cases combined with some comprehensive indices (mean absolute error, mean squared error and standard deviation) were used to test the performance of the proposed method. The results show that the performance of the proposed method is superior to that of existing methods, including the wavelet neural network optimized with the improved butterfly optimization algorithm. In addition, the proposed method has fewer unknown parameters than previous methods, which makes its calculations more convenient. In this study, the effect of the number of training points on the calculation results is also discussed, and the advantage of the proposed method is that only a few training points are needed to achieve high computational accuracy. In addition, as a case study, the proposed method is successfully applied to simulate the water flow in unsaturated soils. The proposed method provides new insight for solving elliptic partial differential equations.

Appendix

A1 Expressions of T _j in Eq. (21) and $\Omega _{ij}$ in Eq. (23)

In Eq. (21),

$$\begin{array}{c}\frac{\partial A({X}_{j})}{\partial x}=-\frac{{h}_{0}({y}_{j})}{b-a}+\frac{{h}_{1}({y}_{j})}{b-a}+\frac{(d-{y}_{j})}{d-c}\left\{\frac{\partial {q}_{0}({x}_{j})}{\partial x}-\left[-\frac{{q}_{0}(a)}{b-a}+\frac{{q}_{0}(b)}{b-a}\right]\right\}\\ +\frac{{y}_{j}-c}{d-c}\left\{\frac{\partial {q}_{1}({x}_{j})}{\partial x}-\left[-\frac{{q}_{1}(a)}{b-a}+\frac{{q}_{1}(b)}{b-a}\right]\right\}\end{array}$$

$$\frac{{\partial }^{2}A({X}_{j})}{\partial {x}^{2}}=\frac{(d-{y}_{j})}{d-c}\left\{\frac{{\partial }^{2}{q}_{0}({x}_{j})}{\partial {x}^{2}}\right\}+\frac{{y}_{j}-c}{d-c}\left\{\frac{{\partial }^{2}{q}_{1}({x}_{j})}{\partial {x}^{2}}\right\}$$

$$\begin{aligned}\frac{\partial A({X}_{j})}{\partial y}&=\frac{(b-{x}_{j})}{b-a}\frac{\partial {h}_{0}({y}_{j})}{\partial y}+\frac{{x}_{j}-a}{b-a}\frac{\partial {h}_{1}({y}_{j})}{\partial y}\\&\quad-\frac{1}{d-c}\left\{{q}_{0}({x}_{j})-\left[\frac{\left(b-{x}_{j}\right)}{b-a}{q}_{0}(a)+\frac{{x}_{j}-a}{b-a}{q}_{0}(b)\right]\right\}\\&\quad+\frac{1}{d-c}\left\{{q}_{1}({x}_{j})-\left[\frac{\left(b-{x}_{j}\right)}{b-a}{q}_{1}(a)+\frac{{x}_{j}-a}{b-a}{q}_{1}(b)\right]\right\}\end{aligned}$$

$$\frac{{\partial }^{2}A({X}_{j})}{\partial {y}^{2}}=\frac{(b-{x}_{j})}{b-a}\frac{{\partial }^{2}{h}_{0}({y}_{j})}{\partial {y}^{2}}+\frac{{x}_{j}-a}{b-a}\frac{{\partial }^{2}{h}_{1}({y}_{j})}{\partial {y}^{2}}$$

In Eq. (23),

$$K({X}_{i},{X}_{j})=\mathrm{exp}\left(-\frac{{\Vert {X}_{i}-{X}_{j}\Vert }^{2}}{{\gamma }^{2}}\right)$$

$$\frac{\partial K\left({X}_{i},{X}_{j}\right)}{\partial x}=-2({x}_{i}-{x}_{j})\mathrm{exp}\left(-\frac{{\Vert {X}_{i}-{X}_{j}\Vert }^{2}}{{\gamma }^{2}}\right)/{\gamma }^{2}$$

$$\frac{\partial K\left({X}_{i},{X}_{j}\right)}{\partial y}=-2({y}_{j}-{y}_{i})\mathrm{exp}\left(-\frac{{\Vert {X}_{i}-{X}_{j}\Vert }^{2}}{{\gamma }^{2}}\right)/{\gamma }^{2}$$

$$\begin{aligned}\frac{{\partial }^{2}K\left({X}_{i},{X}_{j}\right)}{\partial {x}^{2}}&=-2({\gamma }^{2}-2{x}_{j}{}^{2}+4{x}_{j}{x}_{i}-2{x}_{i}{}^{2})\mathrm{exp}\left(-\frac{{\Vert {X}_{i}-{X}_{j}\Vert }^{2}}{{\gamma }^{2}}\right)/{\gamma }^{4} \end{aligned}$$

$$\begin{aligned}\frac{{\partial }^{2}K\left({X}_{i},{X}_{j}\right)}{\partial {y}^{2}}=-2({\gamma }^{2}-2{y}_{j}{}^{2}+4{y}_{j}{y}_{i}-2{y}_{i}{}^{2})\mathrm{exp}\left(-\frac{{\Vert {X}_{i}-{X}_{j}\Vert }^{2}}{{\gamma }^{2}}\right)/{\gamma }^{4} \end{aligned}$$

$$\frac{\partial B({X}_{j})}{\partial x}=-(c-{y}_{j})(d-{y}_{j})\left(a+b-2{x}_{j}\right)$$

$$\frac{\partial B({X}_{j})}{\partial y}=-(a-{x}_{j})(b-{x}_{j})\left(c+d-2{y}_{j}\right)$$

$$\frac{{\partial }^{2}B({x}_{j},{y}_{j})}{\partial {x}^{2}}=2(c-{y}_{j})(d-{y}_{j})$$

$$\frac{{\partial }^{2}B({X}_{j})}{\partial {y}^{2}}=2(a-{x}_{j})(b-{x}_{j})$$

A2 Brief Introductions to the Methods Used for Comparison

ChNN: This method used the Chebyshev neural network to construct a trial solution and used a gradient-based optimization algorithm to learn the connection weight and bias of the network.

WNNPSO: The wavelet neural network was used to construct the trial solution. The particle swarm optimization algorithm was used as the learning algorithm to obtain the connection weights and biases of the wavelet neural network. The particle swarm optimization algorithm is a classic heuristic optimization algorithm pioneered by Kennedy and Eberhart [34] and has been widely used in practice [35].

WNNBOA: The wavelet neural network was used to construct the trial solution. The butterfly optimization algorithm  [36, 37] was used to learn the connection weights and biases of wavelet neural networks.

WNNIBOA: The wavelet neural network was used to construct the trial solution. The improved butterfly optimization algorithm was used to learn the connection weights and biases of wavelet neural networks. Compared with the standard butterfly optimization algorithm, the adjustment parameters in the improved butterfly optimization algorithm change dynamically, which theoretically enhances the algorithm's global optimization capability.

The above four methods all need to determine a large number of parameters (network connection weight and bias).