Inflow forecasting using regularized extreme learning machine: Haditha reservoir chosen as case study

Abstract

For effective water resource management, water budgeting, and optimal release discharge from a reservoir, the accurate prediction of daily inflow is critical. An attempt has been made using artificial intelligence (AI) techniques to enhance water management efficiency in the Haditha-dam reservoir. This case study occasionally suffers from severe drought events and thus causes significant water shortages as well as stopping hydroelectric power stations for several months. Four different approaches were employed for inflow forecasting, namely multiple linear regression (MLR), random forest (RF), extreme learning machine (ELM), and regularized extreme learning machine (RELM). Autocorrelation function (ACF) and partial autocorrelation function (PACF) were used to select the best-lagged variables. The obtained results revealed the superiority of the RELM model compared to other forecast models. The proposed model (RELM) yielded higher prediction accuracy, and its prediction records were similar to the actual values. Moreover, the adopted model achieved a higher correlation of coefficient value (R = 0.955). The regularization approach effectively enhanced the prediction capacity and the generalization ability of the proposed model. On the other hand, the RF model's performance capacity was poor compared to other comparable models due to the overfitting issue. Moreover, the results showed that the PACF (partial autocorrelation function) gave more accurate and realistic predictors than ACF (autocorrelation function) because of its ability to cope with a sudden temporal variation of inflow time series. Overall, the RELM approach provided higher adequacy and tighter confidence in forecasting daily inflow even in noisy data and severe climatic conditions.

Appendix

1.1 Multiple linear regression

At a given time stage, the reservoir inflow is associated with the reservoir inflow that precedes and follows it. Inner interactions between inputs and outputs may operate on a natural system. Multiple linear regression model is a linear equation matching technique for modeling the relationship between the predicted variable and two or more predictors. The MLR can then be used to identify the relationship between dependent and independent variables as a multivariate statistical method which is described in Eq. (23).

$$Y= {\alpha }_{o}+ {\alpha }_{1}{X}_{1}+ {\alpha }_{2}{X}_{2 +\dots }{\alpha }_{\mathrm{j}}{X}_{1}$$

(23)

where Y represent the response variables, ${X}_{1},{X}_{2 },{X}_{n}$ is the independent variables; and ${\alpha }_{o}$,${\alpha }_{1},{\alpha }_{2}\dots {\alpha }_{j}$ are the coefficients of regression, which can be acquired by Eq. (24):

$$e=\sum_{i=1}^{n}(Y-{y}_{i}{)}^{2}$$

(24)

$$\frac{\partial e}{{\partial \alpha }_{o}}=0 ;\quad \frac{\partial e}{{\partial \alpha }_{1}}=0 ;\quad \frac{\partial e}{{\partial \alpha }_{2}}=0 \frac{\partial e}{{\partial \alpha }_{2}}=0$$

whereas the error of estimated and real values of inflow rate is represented by e and ${y}_{i}$, respectively.

1.2 Random forest

Random Forests (RF) is a high-dimensional regression algorithm. This approach is based on trees, where all trees have random selection variables, and the forest is made up of several regression trees and is bundled together (Hameed et al. 2021c, d). The tree is defined as a random subset of variables to determine the prediction outcome. Although the random learning process specifies two essential parameters, the first is the number of trees (ntrees) and the second is the number of variables in each division (mtry). The final decision will be made based on average results following the integration of the individual tree into the ensemble (bagging process). The distance between the bagged trees equals one tree, while the difference is minimized as the relationship between the trees decreases.

The procedure combines with tree growth to estimate random vectors $(\ominus)$ for the regression-based formation of RF to provide numerical values for the tree predictor $h(X,\ominus)$. The mean squared generalization error for any given numerical estimator may be expressed in the Eq. (25).

$${E}_{(X,Y)}=(Y-h{\left(X\right)}^{2})$$

(25)

The RF predictor is achieved by combining an approximation of over j of a single tree. Next, in this context, line up the following theorems:

Theorem 1 By expanding the quantity of trees in the forest, the error will be conveyed by Eq. (26).

$${E}_{\left(X,Y\right)}{(Y{av}_{j}h\left(X,{\ominus}_{j}\right))}^{2}\rightarrow {E}_{X.Y}{(Y-{E}_{\ominus}h\left(X,\ominus\right))}^{2}$$

(26)

The equation in the right hand indicates the error of generalization of the forest. In the same way, the average tree generalization error can be indicated from Eq. (27).

$$PE\times \left(tree\right)={E}_{\ominus}{E}_{X,Y}{(Y-h\left(X,\ominus\right))}^{2}$$

(27)

Theorem 2 If we assume ${E}_{Y}={E}_{X}h(X,\ominus)$ for every $\ominus$, then Eq. (28)

$$PE\left(forest\right)\le \overline{p }.PE(tree)$$

(28)

where, $\overline{p }$ represent the weight correlation.

1.3 Extreme learning machine

Extreme learning machine (ELM) is a modern learning algorithm with a convenient layout, typically made up of three layers: the input, hidden, and output. The hidden layer is one of the most important layers in the ELM scheme, with several nonlinear hidden nodes. The ELM can be mainly defined because the model's internal parameters, such as hidden neurons, do not require tuning. ELM is also referred to as an improved version of conventional ANN, although it can resolve regression issues over a shorter period of time (Deo et al. 2017; Deo and Şahin 2015; Kim et al. 2017). It is very fast because the weights are associated with the hidden layer of the input layer, and the bias values are randomly selected while the output weights are optimally calculated using the Moore–Penrose equation (Huang et al. 2006). Thus, it would result in better performance relative to other forecasting models that can be measured using the ANN methodology (Atiquzzaman and Kandasamy 2015; Ebtehaj et al. 2018; Shiri et al. 2016).

ELM is often recognized as an essential and alternative solution to conventional modeling technologies such as ANN, which are often hindered by various issues such as overfitting, poor convergence, local minimum issues, weaker generalization, longer run, and iterative tuning. Focus on the basic structure of the ELM; the randomly allocated hidden neurons are tuned in such a way that the ELM is powerfully resilient to achieve a global minimum solution, resulting in universal approximation capabilities (Huang and Chen 2007). Mathematically, an ELM model is shown in Eq. (29).

$$\sum_{k=1}^{L}{B}_{k} {g}_{k}\left({\alpha }_{k}.{x}_{k}+{\beta }_{k}\right)={z}_{t},\quad k=1,\dots N \dots$$

(29)

The number of hidden nodes is represented by L, the hidden layer output function is represented by ${g}_{k}\left({\alpha }_{k}.{x}_{k}+{\beta }_{k}\right)$, (${\alpha }_{k}$ and ${\beta }_{k})$ stands for the parameters of hidden nods which are randomly initialized, the weight values linking the kth hidden node(s) with the output node is represented by ${B}_{k}$, and the ELM target is the ${z}_{t}$.

Trial and error in the range of 1–25 can very well establish the number of hidden nodes. The current research has used the hybrid tangent sigmoid transfer function to trigger hidden nodes when predicting ELM model values that depend on the linear activation function obtained from the output layer (Deo and Şahin 2016). The selection of hidden node parameters can be determined arbitrarily if the process does not need precise information, mostly on formation data, nor does the neuron of the hidden layer need to be adjusted according to the sum square error. Accordingly, for any randomly assigned sequence $\{({\alpha }_{k},{\beta }_{k}{{)}^{L}}_{k=1}\}r$ and any continuous target function $f(x)$, Eq. (30) is utilized to estimate and quantify the range of N training samples as follows.

$$\underset{L\to \infty }{\mathrm{lim}}\left|\left|f\left(x\right)-{f}_{L}(x)\right|\right|= \underset{L\to \infty }{\mathrm{lim}}\Vert f\left(x\right)- \sum_{k=1}^{L}{B}_{k} {g}_{k}\left({\alpha }_{k}.{x}_{k}+{\beta }_{k}\right)\Vert =0 \dots$$

(30)

The significant benefits of a non-tuned ELM model are the arbitrary achievement of hidden weight values. Thus, it ends in a zero error and allows the network target weight values (B) for the training data set to be evaluated analytically. It is very worth bearing in mind that the value of the internal transfer function factors (${\alpha }_{k} and {\beta }_{k}$) is specified according to the probability distribution. Ultimately, $Y=GB$ is considered an equivalent to Eq. (30), which can be linearly expressed as explained by Eqs. (31) and (32) (Huang et al. 2006).

$$G\left(\alpha , \beta ,x\right)=\left[\begin{array}{c}g\left({x}_{k}\right)\\ .\\ .\\ .\\ g({x}_{n})\end{array}\right]=\left[\begin{array}{c}{g}_{k}\left({a}_{1}.{x}_{1}+{\beta }_{1}\right)\dots { g}_{L}({a}_{L},{b}_{L},{x}_{1)}\\ .\\ .\\ . \\ {g}_{k}\left({a}_{N}.{x}_{N}+{\beta }_{1}\right)\dots { g}_{L}({a}_{L},{b}_{L},{x}_{1)}\end{array}\right] \dots$$

(31)

and

$$B=\left[\begin{array}{c}{{B}_{1}}^{T}\\ .\\ .\\ .\\ {{B}_{L}}^{T}\end{array}\right]\mathrm{ and}, Y=\left[\begin{array}{c}{{Y}_{1}}^{T}\\ .\\ .\\ .\\ {{Y}_{N}}^{T}\end{array}\right] \dots$$

(32)

whereas the output matrix of the hidden layer is represented by G, and the transpose matrix is represented by T. Subsequently, Eq. 31) can be summarized as shown in Eq. (33).

$$HB=Y \ldots$$

(33)

The lowest norm square of Eq. (32) can be calculated as shown in Eq. (33):

$$\widehat{B}={\check{H}}Y \ldots$$

(34)

The ${\check{H}}$ represents the generalized Moore–Penrose inverse of the Hessian matrix used to measure the output weights of the ELM model. Singular Value Decomposition (SVD) technique is primarily used as an effective solution to the ELM learning process.