Predicting Nitrate Concentration and Its Spatial Distribution in Groundwater Resources Using Support Vector Machines (SVMs) Model

Abstract

In this paper, a support vector machine (SVM) model was developed to predict nitrate concentration in groundwater of Arak plain, Iran. The model provided a tool for prediction of nitrate concentration using a set of easily measurable groundwater quality variables including water temperature, electrical conductivity, groundwater depth, total dissolved solids, dissolved oxygen, pH, land use, and season of the year as input variables. The data set comprised of 160 water samples representing 40 different wells monitored for 1 year. The associated parameters for the optimum SVM model were obtained using a combination of 4-fold cross-validation and grid search technique. The optimum model was used to predict nitrate concentration in Arak plain aquifer. The SVM model predicted nitrate concentration in training and test stage data sets with reasonably high correlation (0.92 and 0.87, respectively) with the measured values and low root mean squared errors of 0.086 and 0.111, respectively. Finally, the map of nitrate concentration in groundwater was prepared for all four seasons using the trained SVM model and a geographic information system (GIS) interpolation scheme and compared with the results with a physics-based (flow and contaminant) model. Overall, the results showed that SVM model could be used as a fast, reliable, and cost-effective method for assessment and predicting groundwater quality.

Appendix A. (SVM Model Background)

Originally developed for binary classification problems, SVMs make use of the hyper-planes to define decision boundaries between the data points of different classes [38]. Then, with the introduction of ε-insensitive loss function, SVM has been extended to solve the regression problems [30]. SVM methods have been mainly employed for regression estimation, so-called support vector regression (SVR) [39]. They were developed from linear classification into nonlinear regression. Nonlinear SVR is based on the concept of mapping data onto high-dimensional feature space through nonlinear mapping (kernel function) and proceeding with linear regression in this space. Suppose the training data set have been taken as “m” vectors {x _i, y _i}, i = 1,…,m where x _i∈R ⁿ is the ith input vector and y _i∈R is its corresponded output. In ε-SVR, which is used in this paper, the aim of learning process is to find a function f(x) as an approximation of the value y(x) that has at most ε deviation from the actually obtained targets y _i for all the training data and at the same time as flat as possible [32, 39]. The objective function of SVM is to minimize the structure risk, which minimizes the empirical error and a regularized term that is called regularized risk function. Also, some error of estimation is taken into account by introducing slack variables ξ and ξ*, as well as the penalty parameter C. The corresponding problem can be equivalent to the following convex constrained quadratic optimization problem:

$$ \begin{array}{cc}\hfill \min imize\kern0.5em {R}_{reg}\left[f\right]=\frac{1}{2}\left\Vert w\left\Vert {}^2+C{\displaystyle \sum_{i=1}^m\left({\xi}_i+{\xi}_i^{\ast}\right),}\right.\right.\hfill & \hfill subject\kern0.5em to\left[\begin{array}{c}\hfill w\cdot \phi \left({x}_i\right)+b-{y}_i\le \varepsilon +{\xi}_i\hfill \\ {}\hfill {y}_i-w\cdot \phi \left({x}_i\right)-b\le \varepsilon +{\xi}_i^{\ast}\hfill \\ {}\hfill {\xi}_i^{\ast },{\xi}_i\ge 0,\kern0.5em i=1,\dots m\hfill \end{array}\right.\hfill \end{array} $$

(A1)

To obtain

$$ \begin{array}{cc}\hfill f(x)={\displaystyle \sum_{i-1}^m\left\langle {w}_i,{\phi}_i\left.(x)\right\rangle +b\right.}\hfill & \hfill \kern0.9em with\hfill \end{array}\kern0.9em W\in {R}^n,b\in R $$

(A2)

where w = {w ₁ w ₂ … w _m} are the SVM weights, ϕ is a kernel function that map input vectors, X = {x ₁ x ₂ … x _m}, into a higher dimensional feature space, 〈w, ϕ〉 denotes the dot product between w and ϕ(x), and b is bias. ‖w‖² is the regularization term which minimizes the complexity of the function f(x) (i.e., the estimated function will always tend to be flat, avoiding over fitting). The second term represents the ε-insensitive loss function depicted in Fig. A1. C >0 is a user-defined constant which determines the trade-off between the flatness of f(x) and the amount up to which deviations larger than ε are tolerated. The ε-insensitive loss function was defined by Vapnik [40] as

Fig. 8

Fig. 8

ε-insensitive loss function

$$ {\left|\xi \right|}_{\varepsilon }={\left|y-f(x)\right|}_{\varepsilon }=\left\{\begin{array}{c}\hfill 0\hfill \\ {}\hfill \left|y-f(x)\right|-\varepsilon \hfill \end{array}\begin{array}{c}\hfill if\left|y-f(X)\right|\le \varepsilon \hfill \\ {}\hfill otherwise\hfill \end{array}\right. $$

(A3)

Usually, Eq. A1 is solved in its dual form using Lagrange multipliers. Transforming this quadratic programming problem to its corresponding dual optimization problem and introducing the kernel function in order to achieve the nonlinearity yields the optimal regression function as [40, 41]

$$ f(X)={\displaystyle \sum_{i=1}^m\left({a}_i^{\ast }-{a}_i\right)K\left({x}_i,x\right)+b} $$

(A4)

where the Lagrange multipliers α_i and α_i* are required to be greater than zero for i = 1,…, m, and K(x _i, x) is a kernel function defined as an inner product in the feature space as follows:

$$ K\left({x}_i,x\right)={\displaystyle \sum_{i-1}^m\varphi {\left({x}_i\right)}_{\cdot}\varphi (x)} $$

(A5)

As a result, the input vectors that correspond to nonzero Lagrangian multipliers, α_i and α_i*, are considered as the support vectors. The SVM model thus is formulated based on these vectors and is guaranteed to have a global, unique, and sparse solution [1].