A novel improved extreme learning machine algorithm in solving ordinary differential equations by Legendre neural network methods
Abstract
This paper develops a Legendre neural network method (LNN) for solving linear and nonlinear ordinary differential equations (ODEs), system of ordinary differential equations (SODEs), as well as classic Emden–Fowler equations. The Legendre polynomial is chosen as a basis function of hidden neurons. A single hidden layer Legendre neural network is used to eliminate the hidden layer by expanding the input pattern using Legendre polynomials. The improved extreme learning machine (IELM) algorithm is used for network weights training when solving algebraic equation systems, and several algorithm steps are summed up. Convergence was analyzed theoretically to support the proposed method. In order to demonstrate the performance of the method, various testing problems are solved by the proposed approach. A comparative study with other approaches such as conventional methods and latest research work reported in the literature are described in detail to validate the superiority of the method. Experimental results show that the proposed Legendre network with IELM algorithm requires fewer neurons to outperform the numerical algorithm in the latest literature in terms of accuracy and execution time.
Keywords
Legendre polynomial Legendre neural network Improved extreme learning machine ODEs Classic Emden–Fowler equation1 Introduction
Many problems encountered in science and engineering, for example, physics, chemistry, biology, mechanics, astronomy, population, resources, economics, and so on, are related to a mathematical model in the form of differential equations. Generally, the analytical expressions of mathematical solutions for practical problems do not exist or are difficult to find. Therefore, it is necessary to study the numerical method of solving differential equations. This means calculating approximate value \(y_{i}\) of exact solution \(y ( x_{i} )\) for differential equations at discrete points \(x_{i}, i = 0,1, \ldots \) in the solution domain.
For a long time, many numerical methods were proposed for solving ODEs [1], including single and multistep methods; singlestep methods include Euler first order method (EM) [2], second order Runge–Kutta (R–K) method inspired by Taylor’s expansion, Suen third order R–K method (SuenRK3), the classic fourthorder R–K method (RK4) [3, 4, 5], etc. In addition, in order to obtain high accuracy, based on the numerical integration method, a lot of linear multistep methods were proposed, such as Adams implicit formulas [6], methods based on Taylor expansion [7], prediction–correction algorithms [8], shooting methods [9], difference methods [10], etc. The numerical methods for solving boundary value problem (BVP) have many mature research results, with high calculation accuracy, but there is a problem that with increasing sample size, the execution time increases rapidly.
With the development of artificial intelligence and computer technology, more and more researchers have developed a keen interest in neural network methods. Neural networks have been used in many fields such as pattern recognition [11], graphics processing [12], risk assessment [13], control systems [14], forecasting [15, 16, 17, 18], and classification [19], showing wide application prospects. Based on the advantages of neural network methods, the use of neural network function approximation capabilities [20, 21, 22] has led to the development of a number of adopted neural network model for solving differential equations. The neural network methods for solving differential equations mainly include the following categories: multilayer perceptron neural network [23, 24, 25, 26, 27, 28], radial basis function neural network [29, 30, 31], multiscale radial basis function neural network [32, 33, 34, 35], cellular neural network [36, 37], finite element neural network [38, 39, 40, 41, 42, 43, 44, 45, 46] and wavelet neural network [28]. The main research focuses on two parts: the construction of the approximate solution and the weights training algorithm.
Approximate solutions of differential equations are often constructed by selecting different activation functions: Meade and Fernandez [47, 48] used a hard limit function as an activation function to construct the neural network model; Lagaris and Likas [23] proposed that multilayer perceptrons can be used to construct approximate solutions; a hybrid technique for constructing the neural network was studied by Ioannis and Tsoulos [49]; Mall and Chakraverty [50] used a Legendre polynomial as an activation function to construct an approximate solution; Xu Liying, Wen Hui and Zeng Zhezhao [51] proposed using a triangular basis function as an activation function to construct approximate solutions for solving ODEs. Regarding the research on network weights training optimization algorithm, we mention Reidmiller and Braun [52] who proposed RPROP algorithm based on local adaptation; Lagaris and Likas [23] proposed using DEevolutionary algorithm to train the weights in the neural network model of partial differential equations; Malek and Shekari Beidokhti [53] presented an optimization algorithm for hybrid neural network model; Rudd and Ferrari [54] analyzed the constrained integral method (CINT) combining the classical Galerkin method with the constrained BP process; Lucie and Peter [55] proposed genetic algorithms for solving a neural network model.
This paper presents a novel Legendre neural network method with improved extreme learning machine algorithm for solving several types of linear or nonlinear differential equations. Candidate solutions are expressed by using Legendre network. With the boundary conditions taken into account, the problem of solving differential equations is transformed into that of nonlinear algebraic equation systems. We call this method for training network weights the improved extreme learning machine algorithm. Convergence analysis, numerical experiments and a comparative study show the superiority of the present method to other classical methods or methods in the recent literature. We believe that the proposed method may be the first to use Legendre neural network model with IELM algorithm in solving differential equations.

It is a single hidden layer neural network—by randomly choosing of the input layer weights, we only need to train the weights of the output layer.

It is easy to implement and runs quickly.

The improved extreme learning machine algorithm is an unsupervised learning algorithm, and we use no optimization technique.

Calculation accuracy is higher than for other numerical methods presented in the recent literature.
The organization of this paper is as follows: we give a description of the problem to be solved in the next section. Section 3 talks about constructing Legendre neural network for approximating and solving ODEs. IELM algorithm for training network weights is proposed and several algorithm steps are summed up in Sect. 4. In Sect. 5, convergence analysis of the proposed Legendre network is verified. We provide many numerical results to verify the effectiveness of the algorithm and its superiority in performance in Sect. 6. Finally, in Sect. 7 we present some conclusions and directions for future research.
2 Description of the problem
We first introduce the general form of the following ordinary differential equations.
2.1 Secondorder ordinary differential equations
2.2 Firstorder system of ordinary differential equations
We know that firstorder ODEs is a particular case of a system of ordinary differential equations (2).
2.3 Higherorder ODEs and higherorder SODE problem
3 Legendre basis function neural network for approximating and solving ODEs
3.1 Legendre basis function neural networks and approximation
In this subsection, employing the recursive properties of Legendre polynomials, we will discuss construction of approximate solutions based on Legendre basis function neural network.
Theorem 1
Proof
Theorem 2
3.2 Legendre basis function neural networks for solving ODEs
By solving the new system equation (21), the unknown weights of the Legendre neural network are obtained.
4 IELM algorithm for training the Legendre neural networks
There are many numerical algorithms for solving system equation (21). In this paper, following the ELM algorithm proposed by Huang Guangbin [57], we use IELM algorithm to train the Legendre network.
Theorem 3
 (I)
If matrixHis a square invertible matrix, then\(\beta = \mathbf{H}^{  1}\mathbf{T}\).
 (II)
If matrixHis rectangular, then\(\beta = \mathbf{H}^{\dagger} \mathbf{T}\), andβis the minimal leastsquares solution of\(\mathbf{H}\beta = \mathbf{T}\), that is, \(\beta = \arg \min \Vert \mathbf{H}\beta  \mathbf{T} \Vert \).
 (III)
IfHis a singular matrix, then\(\beta = \mathbf{H}^{\dagger} \mathbf{T}\), and\(\mathbf{H}^{\dagger} = \mathbf{H}^{T} ( \lambda \mathbf{I} + \mathbf{HH}^{T} )^{  1}\), λis regularization coefficient, which can be set according to a specific instance.
Proof
For the proof of the theorem we refer to the related facts about the generalized inverse matrix in matrix theory [58] and the paper by GuangBin Huang [57].
According to Theorem 2, and as in the article of GuangBin Huang [57], when using extreme learning machine (ELM) algorithm to solve neural network model, that is, when solving \(\mathbf{H}\beta = \mathbf{T}\), the number of hidden neuron nodes must be less than or equal to the sample size, that is, \(N \le M\).
But by matrix analysis theory, if matrix H is rectangular, there exists a β, such that it is the minimal leastsquares solution of \(\mathbf{H}\beta = \mathbf{T}\), that is, \(\beta = \arg \min \Vert \mathbf{H}\beta  \mathbf{T} \Vert \). Here, H is a rectangular matrix, and the number of hidden neuron nodes does not have to be less than or equal to the sample size; we call this improved algorithm for solving \(\mathbf{H}\beta = \mathbf{T}\) the improved extreme learning machine (IELM).
The steps for solving ODEs using Legendre network and IELM algorithm are as follows:
Step 1. Discretize the domain as \(a = x_{0} < x_{1} < \cdots < x_{M} = b\), \(x_{i} = a + \frac{b  a}{M}i, i = 0,1, \ldots,M\), and construct an approximate solution by using Legendre polynomial as an activation function, that is, \(y_{\mathrm{LNN}} ( x ) = \sum_{n = 0}^{N} \beta_{n}P_{n} ( x )\);
Step 2. At discrete points, substitute the approximate solution \(y_{\mathrm{LNN}} ( x )\) and its derivatives into the differential equation and its boundary conditions, and obtain the system equation \(\mathbf{H}\beta = \mathbf{T}\);
Step 3. Solve the system equation \(\mathbf{H}\beta = \mathbf{T}\) by IELM algorithm introduced in Theorem 3, and obtain the network weights \(\beta = \mathbf{H}^{\dagger} \mathbf{T}\), \(\beta = \arg \min \Vert \mathbf{H}\beta  \mathbf{T} \Vert \);
Step 4. Form the approximate solution as \(y_{\mathrm{LNN}} ( x ) = \sum_{n = 0}^{N} \beta_{n}P_{n} ( x ) = \mathbf{P} ( x )\boldsymbol{\beta} \). □
5 Convergence analysis
In this section, we will verify the feasibility and convergence of the LNN method in solving differential equations by proving another theorem.
Theorem 4
Given a standard single layer feedforward neural network with\(n + 1\)hidden nodes and Legendre basis function\(P_{i} ( x ):R \to R, i = 0,1, \ldots,n\), suppose that the approximate solution of onedimensional differential equations is given by (14). If we have any\(m + 1\)distinct samples\(( \mathbf{x},\mathbf{f} )\), for any\(a_{n},b_{n}\)randomly chosen from any intervals ofR, respectively, according to any continuous probability distribution, then the hidden layer output matrixHof Legendre network is invertible and\(\Vert \mathbf{H}\beta  \mathbf{T} \Vert = 0\).
Proof
This means that for any \(a_{n},b_{n}\) randomly chosen from any intervals of R, such as \(a_{n} = 1,b_{n} = 0\), according to any continuous probability distribution, the column vectors of H can be made of full rank with probability one, which validates the above theorem.
Moreover, there exists an \(n \le m\), so that matrix H is rectangular, and given any small positive value \(\varepsilon > 0\) and Legendre activation function \(P_{i} ( x ):R \to R\), for m arbitrary distinct samples \(( \mathbf{x},\mathbf{f} )\), and for any \(a_{n},b_{n}\) randomly chosen from any intervals of R, according to any continuous probability distribution, we have \(\Vert \mathbf{H}_{m \times n}\boldsymbol{\beta}_{n \times 1}  \mathbf{f}_{m \times 1} \Vert < \varepsilon\). □
6 Numerical results and comparative study
Numerical experiments were conducted to verify the effectiveness and superiority of the Legendre network with IELM algorithm. The new scheme was tested on linear differential equations (such as first order, second order ODEs, and SODE) and nonlinear differential equations. A comparative study with other approaches is also described in this section, including traditional methods and latest research works. We will discuss a differential equation appearing in practice (such as Emden–Fowler type equation) to validate the proposed Legendre neural network with IELM algorithm and will show that this method is very encouraging at the end of this section.
All numerical results are obtained using MATLAB R2015a, on a computer with INTEL Core I76500U CPU, 4 GB of memory, 512 GB SSD and WIN10 operating system.
6.1 Experimental results
Example 1
This problem has been solved in \([ 1,2 ]\), and the exact solution is \(y ( x ) = \frac{ ( x + 2 )}{\sin ( x )}\).
Example 2
Example 3
Example 4
It has been solved in \([ 0,1 ]\), with the exact solution being \(y ( x ) = x^{2}\).
Example 5
Example 6
Example 7
Example 8
Example 9
Example 10
Example 11
Mean absolute deviation of test examples with different parameters
Example  m = 100  n = 10  

n = 5  n = 8  n = 10  m = 50  m = 200  m = 500  
Example 1  2.179822e–04  1.124613e–07  6.070510e–08  1.594745e–08  3.258493e–08  1.893462e–08  
Example 2  2.898579e–05  8.788694e–09  1.654316e–10  2.375011e–11  8.614644e–11  5.764976e–11  
Example 3  2.651596e–04  1.352667e–06  2.191930e–08  2.476702e–08  2.04117108  1.950112e–08  
Example 4  1.365455e–15  8.406017e–14  2.180178e–12  9.360722e–13  3.13950e–12  5.474343e–12  
Example 5  4.048192e–15  1.721054e–14  1.911390e–13  1.914236e–13  2.815815e–13  3.639675e–13  
Example 6  1.785691e–06  3.153652e–09  6.852182e–12  4.679648e–12  1.464149e–11  1.074478e–11  
Example 7  1.535907e–05  2.385380e–11  1.783986e–13  8.363132e–14  9.388591e–14  3.149325e–13  
Example 8  9.719036e–04  1.792439e–08  8.837987e–11  1.108446e–10  9.767964e–11  1.867956e–10  
Example 9  \(y_{1}\)  1.011548e–04  1.198037e–08  2.726976e–11  4.126236e–11  3.043601e–11  2.383795e–11 
\(y_{2}\)  1.696858e–04  2.137740e–08  4.481164e–11  1.15183e–10  4.740707e–11  1.311124e–11  
Example 10  \(y_{1}\)  4.814629e–07  7.071364e–12  2.296702e–12  2.432451e–12  2.747277e–12  1.456190e–12 
\(y_{2}\)  2.391016e–07  3.228814e–11  1.792269e–12  8.192227e–13  6.997846e–12  5.358554e–12  
Example 11  5.861896e–02  2.837823e–03  1.038392e–02  2.361003e–02  1.044330e–02  1.047949e–02 
Execution time of test examples with different parameters
Example  m = 100  n = 10  

n = 5  n = 8  n = 10  m = 50  m = 200  m = 500  
Example 1  0.0068  0.0073  0.0073  0.0073  0.0073  0.0077 
Example 2  0.0057  0.0059  0.0065  0.0063  0.0063  0.0064 
Example 3  0.0056  0.0059  0.0061  0.0060  0.0062  0.0068 
Example 4  0.0037  0.0040  0.0040  0.0041  0.0042  0.0048 
Example 5  0.0060  0.0063  0.0066  0.0063  0.0064  0.0071 
Example 6  0.0056  0.0063  0.0062  0.0062  0.0061  0.0067 
Example 7  0.0058  0.0059  0.0062  0.0060  0.0063  0.0067 
Example 8  0.0057  0.0060  0.0061  0.0060  0.0062  0.0075 
Example 9  0.0120  0.0121  0.0121  0.0122  0.0124  0.0132 
Example 10  0.0118  0.0112  0.0114  0.0113  0.0117  0.0123 
Example 11  0.0025  0.0030  0.0037  0.0033  0.0049  0.0082 
6.2 Comparative study
A comparative study with other approaches such as traditional methods and latest research work is described in this subsection to verify the superiority of the proposed method. We first compared our approach with some common traditional methods.
Mean absolute deviation of different methods for firstorder ODEs
ODEs  EM  SuenRK3  RK4  CNN(GD)  CNN(IELM)  LNN 

Example 1  0.009438  7.199659e–08  3.516257e–11  3.038010e–04  0.018023  6.070510e–08 
Example 2  0.006143  2.556570e–08  2.508454e–11  5.306315e–04  0.010317  1.654316e–10 
Example 3  0.003830  6.992166e–08  4.140991e–10  2.502357e–04  0.007621  2.191930e–08 
Example 4  0.005874  4.650619e–09  2.364301e–11  2.463780e–04  0.010018  2.180179e–12 
Execution time of different methods for firstorder ODEs
Parameters of different methods
Methods  Neurons (n)  Sample size (m)  Iterations  Error sum (ε)  Moment (λ) 

EM  –  100  100  –  – 
SuenRK3  –  100  100  –  – 
RK4  –  100  100  –  – 
CNN(GD)  10  100  –  0.01  0.5 
CNN(IELM)  10  100  –  –  – 
LNN  10  100  –  –  – 
Mean absolute deviation of different methods for secondorder ODEs
Mean absolute deviation of different methods for SODE
The execution time for each algorithm in Table 4 is the average time of 30 repetitions, while in Tables 7 and 9, we averaged over 100 times (results are in seconds).
By comparing with traditional methods, we testified the superiority of the new method both in terms of calculation accuracy and execution time. In order to further prove the superiority of the proposed method, a comparison with the latest reported methods is done. The following three ODEs chosen for testing the proposed method are boundary problems.
Example 12
Example 13
The sampling parameter in this test experiment is \(m = 10\), the result is shown in Fig. 9(b). By comparison, the maximum error of BeNN method in [60] is 7.3e–9, and the maximum error of the proposed LNN method is 5.2e–12, so it is easy to seen from Fig. 9(b) that LNN method can obtain higher accuracy solution than BeNN method in [60]. Considering the method given by [61], the maximum error is 3.5e–2 with \(m = 50\), while, using LNN method, we are able to obtain a higher accuracy with maximum error 2.4e–13 by \(n = 10\) neurons, which also fully validates the superiority of LNN method with IELM algorithm.
Example 14
6.3 Classic Emden–Fowler equation
Many problems in science and engineering can be modelled by Emden–Fowler equation. A lot of attention has been focused on the numerical solution of Emden–Fowler type equation. In this subsection, we will apply the proposed alternative approach of Legendre neural network (LNN) with IELM algorithm to solve classic Emden–Fowler equation.
Example 15
Exact and LNN neural network results
Input points \(x_{k}\)  Exact results  LNN results  Absolute error 

0  1  1  0 
0.1  0.9966699984131  0.9966699984129  1.8e–13 
0.2  0.9867198985254  0.9867198985252  2.2e–13 
0.3  0.9702688457452  0.9702688457450  2.3e–13 
0.4  0.9475135272247  0.9475135272245  2.3e–13 
0.5  0.9187253698655  0.9187253698653  2.3e–13 
0.6  0.8842466786034  0.8842466786032  2.2e–13 
0.7  0.8444857748514  0.8444857748511  2.2e–13 
0.8  0.7999112103978  0.7999112103976  2.1e–13 
0.9  0.7510451462491  0.7510451462489  2.0e–13 
1  0.6984559986366  0.6984559986364  1.8e–13 
7 Conclusions
In this paper, we have presented a novel Legendre neural network to solve several linear or nonlinear ODEs. A Legendre polynomial was chosen as a basis function of the hidden neurons. We used Legendre polynomials to eliminate the hidden layer of the network by expanding the input pattern. An improved extreme learning machine (IELM) algorithm was used for network weights training when solving the algebraic equation systems. Convergence analysis has proved the feasibility of this method. The accuracy of the proposed method has been examined by solving a lot of testing examples, and the results obtained by the proposed method have been compared with the exact solution. We found the presented method to be better. A comparative study has fully validated the superiority of the new proposed method over other numerical algorithms published in the latest literature. An application of the approach to solve the classic Emden–Fowler equation also shows the feasibility and applicability of our method. From the presented investigation we can see that the LNN neural network with IELM algorithm is straightforward, easily implementable and has higher accuracy when solving ODEs.
In addition, the neural network for solving ODEs and PDEs has been discussed a lot and still has some potential to work on. The recent research articles such as [63, 64, 65, 66] have studied using neural network method to solve several fractional differential equations (FDEs). We have never dealt with the numerical solution of FDEs using neural network method. This will become an important research direction for us in the future. As mentioned in many articles, a variety of phenomena in astrophysics and mathematical physics can be described by Emden–Fowler equations, so this differential equation will also become a research direction for us in the future. If we consider only one type of orthogonal polynomial, there are some published papers as [67, 68], hybrid methods may also be a new research direction.
Notes
Acknowledgements
The authors sincerely thank the reviewers for their careful reading and valuable comments, which improved the quality of this paper.
Authors’ contributions
All authors contributed to the draft of the manuscript, all authors read and approved the final manuscript.
Funding
This work was supported by the National Natural Science Foundation of China under Grants 61375063, 61271355, 11301549 and 11271378.
Competing interests
The authors declare that they have no competing interests.
References
 1.Han, X.: Numerical Analysis. Higher Education Press, Beijing (2011) (in Chinese) Google Scholar
 2.Zhang, Q.: Singlestep method for solving the initial value problem of ordinary differential equations. Bull. Sci. Tech. 28(2), 7–9 (2012) Google Scholar
 3.Wang, B.: Numerical solution of differential equations and Matlab implementation. J. Tongling Vocat. Tech. Coll. 10(3), 95–97 (2011) Google Scholar
 4.Wang, P., Yuan, H., Liu, P., et al.: Finite difference method and implicit Runge–Kutta method for solving Burgers equation. J. Changchun Univ. Sci. Technol. 36(1), 158–160 (2013) Google Scholar
 5.Deng, C., Dai, Z., Jiang, S.: Higherorder explicit index Runge–Kutta method. J. Beijing Univ. Chem. Technol. 40(5), 123–127 (2013) Google Scholar
 6.Li, M., Chen, H., Zhang, Z. Fifthorder Adams scheme for solving first order ODEs. J. Shaoguan Univ. 34(12), 14–17 (2013) Google Scholar
 7.Huang, Z., Hu, Z., Wang, C..: A study on construction for linear multistep methods based on Taylor expansion. Adv. Appl. Math. 4(4), 343–356 (2015) Google Scholar
 8.Liu, D.: Contrast test of predictorcorrector method for fourthorder Adams–Bashforth combination formula. J. Xichang Coll. 26(3), 43–46 (2012) Google Scholar
 9.Le, A.: Shooting method in the application of ordinary difference equations boundary value problem. Sci. Mosaic 5, 247–249 (2011) Google Scholar
 10.Li, S.: Finite difference method for ordinary differential equations and its simple application. Anhui University (2010) Google Scholar
 11.Abdulla, M.B., Costa, A.L., Sousa, R.L.: Probabilistic identification of subsurface gypsum geohazards using artificial neural networks. Neural Comput. Appl. 29(12), 1377–1391 (2018) Google Scholar
 12.Jiao, Y., Pan, X., Zhao, Z., et al.: Learning sparse partial differential equations for vectorvalued images. Neural Comput. Appl. 29(11), 1205–1216 (2018) Google Scholar
 13.Wang, Y., Liu, M., Bao, Z., et al.: Stacked sparse autoencoder with PCA and SVM for databased line trip fault diagnosis in power systems. Neural Comput. Appl. (2018). https://doi.org/10.1007/s0052101834905 Google Scholar
 14.Qiao, J., Zhang, W.: Dynamic multiobjective optimization control for wastewater treatment process. Neural Comput. Appl. 29(11), 1261–1271 (2018) Google Scholar
 15.Armaghani, D.J., Hasanipanah, M., Mahdiyar, A., et al.: Airblast prediction through a hybrid genetic algorithm—ANN model. Neural Comput. Appl. 29(9), 619–629 (2018) Google Scholar
 16.HernándezTravieso, J.G., RaveloGarcía, A.G., AlonsoHernández, J.B., et al.: Neural networks fusion for temperature forecasting. Neural Comput. Appl. (2018). https://doi.org/10.1007/s0052101834500 Google Scholar
 17.Hou, M., Liu, T., Yang, Y.: A new hybrid constructive neural network method for impacting and its application on tungsten price prediction. Appl. Intell. 47(1), 28–43 (2017) Google Scholar
 18.Xi, L., Hou, M., Lee, M., et al.: A new constructive neural network method for noise processing and its application on stock market prediction. Appl. Soft Comput. 15, 57–66 (2014) Google Scholar
 19.Dwivedi, A.K.: Artificial neural network model for effective cancer classification using microarray gene expression data. Neural Comput. Appl. 29(12), 1545–1554 (2018) Google Scholar
 20.Hou, M.: Han, X.: Constructive approximation to multivariate function by decay RBF neural network. IEEE Trans. Neural Netw. 21(9), 1517–1523 (2010) Google Scholar
 21.Hou, M., Han, XH.: The multidimensional function approximation based on constructive wavelet RBF neural network. Appl. Soft Comput. 11(2), 2173–2177 (2011) Google Scholar
 22.Hou, M., Han, X.: Multivariate numerical approximation using constructive L2(R) RBF neural network. Neural Comput. Appl. 21(1), 25–34 (2012) Google Scholar
 23.Lagaris, I.E., Likas, A.C.: Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw. 9(5), 987–1000 (1998) Google Scholar
 24.He, S., Reif, K., Unbehauen, R.: Multilayer networks for solving a class of partial differential equations. Neural Netw. 13(3), 385–396 (2000) Google Scholar
 25.MaiDuy, N., TranCong, T.: Numerical solution of differential equations using multiquadric radial basis function networks. Neural Netw. 14(2), 185–199 (2001) zbMATHGoogle Scholar
 26.Chua, L.O., Yang, L.: Cellular neural networks: theory. IEEE Trans. Circuits Syst. 35, 1257–1272 (1988) MathSciNetzbMATHGoogle Scholar
 27.Ramuhalli, P., Udpa, L., Udpa, S.S.: Finite element neural networks for solving differential equations. IEEE Trans. Neural Netw. 16(6), 1381–1392 (2005) Google Scholar
 28.Li, X., Ouyang, J., Li, Q., Ren, J.: Integration wavelet neural network for steady convection dominated diffusion problem. In: 3rd International Conference on Information and Computing, vol. 2, pp. 109–112 (2010) Google Scholar
 29.Li, J., Luo, S., Qi, Y., Huang, Y.: Numerical solution of differential equations by radial basis function neural networks. In: Proc. Int. Jt Conf. Neural Netw, vol. 1, pp. 773–777 (2002) Google Scholar
 30.Moody, J.E., Darken, C.: Fast learning in networks of locally tuned processing units. Neural Comput. 1(2), 281–294 (1989) Google Scholar
 31.Esposito, A., Marinaro, M., Oricchio, D., Scarpetta, S.: Approximation of continuous and discontinuous mappings by a growing neural RBFbased algorithm. Neural Netw. 13(6), 651–665 (2000) Google Scholar
 32.Park, J., Sandberg, I.W.: Approximation and radial basis function networks. Neural Comput. 5, 305–316 (1993) Google Scholar
 33.Haykin, S.: Neural Networks: A Comprehensive Foundation. Pearson Education, Singapore (2002) zbMATHGoogle Scholar
 34.Franke, R.: Scattered data interpolation: tests of some methods. Math. Comput. 38(157), 181–200 (1982) MathSciNetzbMATHGoogle Scholar
 35.MaiDuy, N., TranCong, T.: Approximation of function and its derivatives using radial basis function networks. Neural Netw. 27(3), 197–220 (2003) zbMATHGoogle Scholar
 36.Manganaro, G., Arena, P., Fortuna, L.: Cellular neural networks: chaos. In: Complexity and VLSI Processing, pp. 44–45. Springer, Berlin (1999) zbMATHGoogle Scholar
 37.Chedhou, J.C., Kyamakya, K.: Solving stiff ordinary and partial differential equations using analog computing based on cellular neural networks. ISAST Trans. Comput. Intell. Syst. 4(2), 213–221 (2009) Google Scholar
 38.Takeuchi, J., Kosugi, Y.: Neural network representation of the finite element method. Neural Netw. 7(2), 389–395 (1994) Google Scholar
 39.Beltzer, A.I., Sato, T.: Neural classification of finite elements. Comput. Struct. 81(24–25), 2331–2335 (2003) Google Scholar
 40.Topping, B.H.V., Khan, A.I., Bahreininejad, A.: Parallel training of neural networks for finite element mesh decomposition. Comput. Struct. 63(4), 693–707 (1997) zbMATHGoogle Scholar
 41.Manevitz, L., Bitar, A., Givoli, D.: Neural network time series forecasting of finiteelement mesh adaptation. Neurocomputing 63, 447–463 (2005) Google Scholar
 42.Jilani, H., Bahreininejad, A., Ahmadi, M.T.: Adaptive finite element mesh triangulation using selforganizing neural networks. Adv. Eng. Softw. 40(11), 1097–1103 (2009) zbMATHGoogle Scholar
 43.Arndt, O., Barth, T., Freisleben, B., Grauer, M.: Approximating a finite element model by neural network prediction for facility optimization in groundwater engineering. Eur. J. Oper. Res. 166(3), 769–781 (2005) zbMATHGoogle Scholar
 44.Koroglu, S., Sergeant, P., Umurkan, N.: Comparison of analytical, finite element and neural network methods to study magnetic shielding. Simul. Model. Pract. Theory 18(2), 206–216 (2010) Google Scholar
 45.Deng, J., Yue, Z.Q., Tham, L.G., Zhu, H.H.: Pillar design by combining finite element methods, neural networks and reliability: a case study of the Feng Huangshan copper mine, China. Int. J. Rock Mech. Min. Sci. 40(4), 585–599 (2003) Google Scholar
 46.Ziemianski, L.: Hybrid neural network finite element modeling of wave propagation in infinite domains. Comput. Struct. 81(8–11), 1099–1109 (2003) Google Scholar
 47.Meade, A.J. Jr., Fernandez, A.A.: The numerical solution of linear ordinary differential equations by feedforward neural networks. Math. Comput. Model. 19, l–25 (1994) MathSciNetzbMATHGoogle Scholar
 48.Meade, A.J. Jr., Fernandez, A.A.: Solution of nonlinear ordinary differential equations by feedforward neural networks. Math. Comput. Model. 20(9), 19–44 (1994) MathSciNetzbMATHGoogle Scholar
 49.Tsoulos, I.G., Gavrilis, D., Glavas, E.: Solving differential equations with constructed neural networks. Neurocomputing 72(10–12), 2385–2391 (2009) Google Scholar
 50.Mall, S., Chakraverty, S.: Application of Legendre neural network for solving ordinary differential equations. Appl. Comput. 43, 347–356 (2016) Google Scholar
 51.Xu, L.Y., Hui, W., Zeng, Z.Z.: The algorithm of neural networks on the initial value problems in ordinary differential equations. In: Industrial Electronics and Applications. 2007. Iciea 2007. IEEE Conference on, pp. 813–816 IEEE New York (2007) Google Scholar
 52.Reidmiller, M., Braun, H.: A direct adaptive method for faster back propagation learning: the RPROP algorithm. In: Proceedings of the IEEE International Conference on Neural Networks, pp. 586–591 (1993) Google Scholar
 53.Malek, A., Beidokhti, R.S.: Numerical solution for high order differential equations using a hybrid neural networkoptimization method. Appl. Math. Comput. 183(1), 260–271 (2006) MathSciNetzbMATHGoogle Scholar
 54.Rudd, K., Ferrari, S.: A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks. Neurocomputing 155, 277–285 (2015) Google Scholar
 55.Aarts, L.P., Veer, P.V.: Neural network method for partial differential equations. Neural Process. Lett. 14(3), 261–271 (2001) zbMATHGoogle Scholar
 56.Chaquet, J.M., Carmona, E.: Solving differential equations with Fourier series and evolution strategies. Appl. Soft Comput. 12, 3051–3062 (2012) Google Scholar
 57.Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006) Google Scholar
 58.Dai, H., Matrix Theory. Science Press, Beijing (2001) Google Scholar
 59.Mall, S., Chakraverty, S.: Application of Legendre neural network for solving ordinary differential equations. Appl. Soft Comput. 43, 347–356 (2016) Google Scholar
 60.Sun, H., Hou, M., Yang, Y., et al.: Solving partial differential equation based on Bernstein neural network and extreme learning machine algorithm. Neural Process. Lett. (2018). https://doi.org/10.1007/s1106301899118 Google Scholar
 61.Yazdi, H.S., Pakdaman, M., Modaghegh, H.: Unsupervised kernel least mean square algorithm for solving ordinary differential equations. Neurocomputing 74, 2062–2071 (2011) Google Scholar
 62.Chowdhury, M.S.H., Hashim, I.: Solutions of Emden–Fowler equations by homotopyperturbation method. Nonlinear Anal., Real World Appl. 10, 104–115 (2009) MathSciNetzbMATHGoogle Scholar
 63.ZunigaAguilar, C.J., CoronelEscamilla, A., GomezAguilar, J.F., et al.: New numerical approximation for solving fractional delay differential equations of variable order using artificial neural networks. Eur. Phys. J. Plus 133(2), 75 (2018) Google Scholar
 64.ZunigaAguilar, C.J., RomeroUgalde, H.M., GomezAguilar, J.F., et al.: Solving fractional differential equations of variableorder involving operators with MittagLeffler kernel using artificial neural networks. Chaos Solitons Fractals 103, 382–403 (2017) MathSciNetzbMATHGoogle Scholar
 65.Rostami, F., Jafarian, A.: A new artificial neural network structure for solving highorder linear fractional differential equations. Int. J. Comput. Math. 95(3), 528–539 (2018) MathSciNetzbMATHGoogle Scholar
 66.Pakdaman, M., Ahmadian, A., Effati, S., et al.: Solving differential equations of fractional order using an optimization technique based on training artificial neural network. Appl. Math. Comput. 293, 81–95 (2017) MathSciNetGoogle Scholar
 67.Mall, S., Chakraverty, S.: Chebyshev neural network based model for solving Lane–Emden type equations. Appl. Math. Comput. 247, 100–114 (2014) MathSciNetzbMATHGoogle Scholar
 68.Chaharborj, S.S., Chaharborj, S.S., Mahmoudi, Y.: Study of fractional order integrodifferential equations by using Chebyshev neural network. J. Math. Stat. 13(1), 1–13 (2017) Google Scholar
Copyright information
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.