A Methodology for Sensitivity Analysis Based on Regression: Applications to Handle Uncertainty in Natural Resources Characterization

Abstract

Uncertainty in a natural resource model represents risk that should be minimized for improved management. Natural resources are components of complex earth science systems, which can be exhaustively sampled, numerically modeled with Monte Carlo simulation, or both, to understand their underlying nature. A numerical model represents relationships between a response variable and predictor variables. Uncertainty in a response variable can be observed directly, but understanding the importance of each predictor variable requires further post-processing of the numerical model. A methodology of local sensitivity analysis, based on linear and quadratic regression models, is developed to help understand the uncertainty contribution of each predictor variable to the response model. Sensitivity coefficients, predicted response values, and summary statistics with model utility tests for the regression models are evaluated. The importance of standardized sensitivity coefficients and other measures are developed. Standardized sensitivity coefficients represent the contribution of uncertainty in the predictor variables to uncertainty in model response. Results of the sensitivity analysis are visually summarized in the form of extended tornado charts. The proposed methodology is applied to representative petroleum and mining case studies. The methodology is robust, efficient, descriptive, and straightforward. Understanding of the contribution of each predictor variable to the response variable is useful for minimization of model response uncertainty, decision-making, and further study.

Appendices

Appendix 1—Derivation of Sensitivity Coefficients from Linear and Quadratic Regression Models

The general expression of the regression model with assumed residual properties and algorithm for derivation of the regression coefficients are defined in Eqs. (35)–(44) (Johnson and Wichern 2007). A regression model is a probabilistic model whose coefficients must be estimated. The probabilistic part is depicted through error terms ε added to the deterministic model. The deviations from the mean are chosen as data values to mitigate the influence of the measurement units and to help derivation of the sensitivity coefficients.

$$ {\mathbf{y}}' = {\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}} $$

(35)

$$ {\mathbf{y}}' = {\mathbf{y}} - {\mathbf{1}} \cdot \bar{y} $$

(36)

$$ {\mathbf{x}}' = {\mathbf{x}} - {\mathbf{1}} \cdot {\bar{\mathbf{x}}} $$

(37)

$$ {\bar{\mathbf{x}}} = \left[ {\bar{x}_{1} \ldots \bar{x}_{K} } \right]_{1 \times K} $$

(38)

$$ \bar{y} = \sum\limits_{i = 1}^{n} {\frac{{y_{i} }}{n}} ,\quad \lim \left( {\bar{y}} \right)\mathop = \limits_{n \to \infty } E\left\{ Y \right\} = m_{Y} $$

(39)

$$ \bar{x}_{k} = \sum\limits_{i = 1}^{n} {\frac{{x_{k,i} }}{n}} ,\quad \lim \left( {\bar{x}_{k} } \right)\mathop = \limits_{n \to \infty } E\left\{ {X_{k} } \right\} = m_{{X_{k} }} ,\quad k = 1, \ldots ,K $$

(40)

$$ {\varvec{\upvarepsilon}}\sim N({\mathbf{0}},\,\sigma_{\varepsilon }^{2} \cdot {\mathbf{I}}) = \left\{ {\begin{array}{l} {{\varvec{\upvarepsilon}}\;{\text{follows}}\;{\text{normal}}\;{\text{distribution}}} \\ {E\left\{ {\varvec{\upvarepsilon}} \right\} = {\mathbf{0}}} \\ {{\text{COV}}\left\{ {\varvec{\upvarepsilon}} \right\} = \sigma_{\varepsilon }^{2} \cdot {\mathbf{I}}} \\ \end{array} } \right. $$

(41)

$$ {\hat{\mathbf{y}}}' = {\mathbf{x}}' \cdot {\varvec{\hat \upbeta}} $$

(42)

$$ {\varvec{\hat \upvarepsilon}} = {\mathbf{y}}' - {\hat{\mathbf{y}}}' $$

(43)

$$ \hbox{min} \left( {\text{RSS}} \right) = \hbox{min} \left( {{\varvec{\hat \upvarepsilon}}^{T} \cdot {\varvec{\hat \upvarepsilon}}} \right)\quad \Rightarrow \quad {\varvec{\hat \upbeta}} = \left[ {\left[ {{\mathbf{x}}'} \right]^{T} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{T} \cdot {\mathbf{y}}' $$

(44)

where y and x are the n × 1 vector and n × K matrix containing original data values of a model response Y and input variables X _k , k = 1, …, K, respectively; n is the number of data; y′ and x′ are the n x 1 vector and n × K matrix, respectively, containing modified data values of the model response and input variables; $\bar{y}$ is the average of the model response; ${\bar{\mathbf{x}}}$ is the 1 × K row vector of average values of the input variables; 1 is the n × 1 column vector consisting of 1 s; m _Y and $m_{{X_{k} }}$ are the expected mean values of the model response and input variables, respectively; β is the K × 1 column vector consisting of regression coefficients; ε is the n × 1 column vector of normally distributed independent errors or residuals with zero mean and constant variance σ ² _ε ; ${\hat{\mathbf{y}}}'$, ${\varvec{\hat \upbeta}}$, and ${\varvec{\hat \upvarepsilon}}$ are the estimates of model response, regression coefficients, and errors, respectively; and RSS is the residual sum of squares, which is used to estimate the regression coefficients. Regression coefficients that give the smallest RSS, in other words, the smallest deviations of the regression model from the actual data, are optimal regression coefficients. The form and properties of the linear and quadratic regression models are explained below.

The linear regression model or 1st-order regression model is the simplest member of the regression models family that can be used to fit the data. Linear regression model takes form of Eq. (45). The related regression model parameters for matrix form are explained in the subsequent Eqs. (46)–(48). We might draw the similarity between the simple kriging and proposed regression model, if spatial context of the data is ignored (Journel and Huijbregts 1978). Unrealistically influential data values and outliers might be removed from the data set to reduce bias in the sensitivity study.

$$ \hat{y}_{i}^{(1)} = \bar{y} + \sum\limits_{k = 1}^{K} {\hat{\beta }_{k}^{(1)} \cdot \left( {x_{k,i} - \bar{x}_{k} } \right)} ,\quad i = 1, \ldots ,n $$

(45)

$$ {\hat{\mathbf{y}}}'^{(1)} = \left[ {\begin{array}{l} {\hat{y}_{1}^{(1)} - \bar{y}} \\ \cdots \\ {\hat{y}_{n}^{(1)} - \bar{y}} \\ \end{array} } \right]_{n \times 1} $$

(46)

$$ {\mathbf{x}}'^{(1)} = \left[ {\begin{array}{lll} {x_{1,1} - \bar{x}_{1} } & \cdots & {x_{K,1} - \bar{x}_{K} } \\ \cdots & {} & \cdots \\ {x_{1,n} - \bar{x}_{1} } & \cdots & {x_{K,n} - \bar{x}_{K} } \\ \end{array} } \right]_{n \times K} $$

(47)

$$ {\varvec{\hat \upbeta}}^{({1})} = \left[ {\begin{array}{l} {\hat{\beta }_{1}^{\left( 1 \right)} } \\ \cdots \\ {\hat{\beta }_{K}^{\left( 1 \right)} } \\ \end{array} } \right]_{K \times 1} $$

(48)

The quadratic regression model takes form of Eq. (49), which is similar to the linear regression model except the interaction terms (product of two input variables). The re-occurring interaction terms between any two input variables are omitted in the quadratic regression model to avoid matrix singularity in the computation of the regression coefficients in Eq. (44). Note that interaction terms are treated as additional input variables and, in general, the form of the quadratic regression model does not differ much from the linear one. The parameters of the quadratic regression model for matrix form are explained in Eqs. (50)–(54).

$$ \begin{aligned} \hat{y}_{i}^{(2)} = & \bar{y} + \sum\limits_{k = 1}^{K} {\hat{\beta }_{k}^{(2)} \cdot \left( {x_{k,i} - \bar{x}_{k} } \right)} \\ & + \sum\limits_{k = 1}^{K} {\sum\limits_{k' = k}^{K} {\hat{\beta }_{kk'}^{(2)} \cdot \left( {\left( {x_{k,i} - \bar{x}_{k} } \right) \cdot \left( {x_{k',i} - \bar{x}_{k'} } \right) - \overline{{\left( {x_{k,i} - \bar{x}_{k} } \right) \cdot \left( {x_{k',i} - \bar{x}_{k'} } \right)}} } \right)} } ,\quad i = 1, \ldots ,n \\ \end{aligned} $$

(49)

$$ {\hat{\mathbf{y}}}'^{(2)} = \left[ {\begin{array}{l} {\hat{y}_{1}^{(2)} - \bar{y}} \\ \cdots \\ {\hat{y}_{n}^{(2)} - \bar{y}} \\ \end{array} } \right]_{n \times 1} $$

(50)

$$ \begin{aligned} {\mathbf{x}}'^{(2)} = & \left[ {\begin{array}{*{20}c} {x_{1,1} - \bar{x}_{1} } & \cdots & {x_{K,1} - \bar{x}_{K} } \\ \cdots & {} & \cdots \\ {x_{1,n} - \bar{x}_{1} } & \cdots & {x_{K,n} - \bar{x}_{K} } \\ \end{array} } \right. \\ & \quad \quad \quad \quad \quad \quad \begin{array}{*{20}c} {\left( {x_{1,1} - \bar{x}_{1} } \right) \cdot \left( {x_{1,1} - \bar{x}_{1} } \right) - \overline{{\left( {x_{1,1} - \bar{x}_{1} } \right) \cdot \left( {x_{1,1} - \bar{x}_{1} } \right)}} } & \cdots \\ {} \cdots {} \\ {\left( {x_{1,n} - \bar{x}_{1} } \right) \cdot \left( {x_{1,n} - \bar{x}_{1} } \right) - \overline{{\left( {x_{1,n} - \bar{x}_{1} } \right) \cdot \left( {x_{1,n} - \bar{x}_{1} } \right)}} } & \cdots \\ \end{array} \\ & \left. {\quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \begin{array}{*{20}c} {\left( {x_{K,1} - \bar{x}_{K} } \right) \cdot \left( {x_{K,1} - \bar{x}_{K} } \right) - \overline{{\left( {x_{K,1} - \bar{x}_{K} } \right) \cdot \left( {x_{K,1} - \bar{x}_{K} } \right)}} } \\ \cdots \\ {\left( {x_{K,n} - \bar{x}_{K} } \right) \cdot \left( {x_{K,n} - \bar{x}_{K} } \right) - \overline{{\left( {x_{K,n} - \bar{x}_{K} } \right) \cdot \left( {x_{K,n} - \bar{x}_{K} } \right)}} } \\ \end{array} } \right]_{n \times (K + 3) \cdot K/2} \\ \end{aligned} $$

(51)

$$ {\varvec{\hat \upbeta}}^{({2})} = \left[{\begin{array}{*{20}c} {\hat{\beta }_{1}^{\left( 2 \right)} } \\\cdots \\ {\hat{\beta }_{K}^{\left( 2 \right)} } \\ {\hat{\beta}_{11}^{\left( 2 \right)} } \\ \cdots\\ {\hat{\beta }_{KK}^{\left( 2 \right)} } \\\end{array} } \right]_{(K + 3) \cdot K/2 \times 1} $$

(52)

$$ \overline{{\left( {x_{k,i} - \bar{x}_{k} } \right) \cdot \left( {x_{k',i} - \bar{x}_{k'} } \right)}} = \sum\limits_{i = 1}^{n} {\frac{{\left( {x_{k,i} - \bar{x}_{k} } \right) \cdot \left( {x_{k',i} - \bar{x}_{k'} } \right)}}{n},\quad k,k' = 1, \ldots ,n} $$

(53)

$$ \lim \left( {\overline{{\left( {x_{k,i} - \bar{x}_{k} } \right) \cdot \left( {x_{k',i} - \bar{x}_{k'} } \right)}} } \right)\mathop = \limits_{n \to \infty } {\text{COV}}\left\{ {X_{k} ,X_{k'} } \right\},\quad k,k' = 1, \ldots ,n $$

(54)

By applying definition of the sensitivity coefficients to the regression models, we would get the following relationship between sensitivity and regression coefficients as shown in Eqs. (55)–(57).

$$ \mu_{k}^{(1)} = \left. {\,\frac{{\partial Y^{(1)} }}{{\partial X_{k} }}} \right|_{{X_{l} = m_{{X_{l} }} ,\forall l \ne k}} = \beta_{k}^{(1)} $$

(55)

$$ \mu_{k}^{(2)} = \left. {\,\frac{{\partial Y^{(2)} }}{{\partial X_{k} }}} \right|_{{X_{l} = m_{{X_{l} }} ,\forall l \ne k}} = \beta_{k}^{(2)} $$

(56)

$$ \mu_{kk'}^{(2)} = \left. {\,\frac{{\partial^{2} Y^{(2)} }}{{\partial X_{k} \cdot \partial X_{k'} }}} \right|_{{X_{l} = m_{{X_{l} }} ,\forall l \ne k,k'}} = \left\{ {\begin{array}{*{20}c} {\beta_{kk'}^{(2)} ,\quad k \ne k'} \\ {2\beta_{kk'}^{(2)} ,\quad k = k} \\ \end{array} } \right. $$

(57)

Appendix 2—Derivation of 1st- and 2nd-Order Moments of Regression Model Parameters

Derivation of Mean and Variance of the Estimated Regression Errors

In the following derivations, the matrix x′ of input variables’ deviations from the mean is treated as a constant.

$$ \begin{aligned} E\left\{ {{\hat{\varvec{\upvarepsilon }}}}\right\}\, & = E\left\{ {{\mathbf{y}}' - {\hat{\mathbf{y}}}'}\right\} = E\left\{ {{\mathbf{y}}' - {\mathbf{x}}' \cdot {\hat{\varvec{\upbeta }}}} \right\} \\ & = E\left\{ {{\mathbf{y}}'- {\mathbf{x}}' \cdot \left[ {\left[ {{\mathbf{x}}'}\right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{y}}'}\right\} \\ & = {\mathbf{y}}'\left[ {{\mathbf{I}} - {\mathbf{x}}'\cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot{\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'}\right]^{\text{T}} } \right] \\ \end{aligned} $$

(58)

$$ VAR\left\{ {{\varvec{\hat \upvarepsilon}}} \right\}\, = E\left\{ {\left[ {{\mathbf{y}}' - {\hat{\mathbf{y}}}' - E\left\{ {{\mathbf{y}}' - {\hat{\mathbf{y}}}'} \right\}} \right] \cdot \left[ {{\mathbf{y}}' - {\hat{\mathbf{y}}}' - E\left\{ {{\mathbf{y}}' - {\hat{\mathbf{y}}}'} \right\}} \right]^{\text{T}} } \right\} = E\left\{ {\left[ {{\mathbf{y}}' - {\mathbf{x}}' \cdot {\varvec{\hat \upbeta}} - E\left\{ {{\mathbf{y}}' - {\mathbf{x}}' \cdot {\varvec{\hat \upbeta}}} \right\}} \right] \cdot \left[ {{\mathbf{y}}' - {\mathbf{x}}' \cdot {\varvec{\hat \upbeta}} - E\left\{ {{\mathbf{y}}' - {\mathbf{x}}' \cdot {\varvec{\hat \upbeta}}} \right\}} \right]^{\text{T}} } \right\} = E\left\{ {\left[ {{\mathbf{y}}' - {\mathbf{x}}' \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{y}}'} \right] \cdot \left[ {{\mathbf{y}}' - {\mathbf{x}}' \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{y}}'} \right]^{\text{T}} } \right\} = \left[ {{\mathbf{I}} - {\mathbf{x}}' \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} } \right] \cdot VAR\left\{ {{\mathbf{y}}'} \right\} \cdot \left[ {{\mathbf{I}} - {\mathbf{x}}' \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} } \right]^{\text{T}} = \sigma^{2}_{\varepsilon} \cdot \left[ {{\mathbf{I}} - {\mathbf{x}}' \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} } \right] $$

(59)

Derivation of Mean and Variance of the Model Response

$$ E\left\{ {{\mathbf{y}}'} \right\} = E\left\{ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}}} \right\} = E\left\{ {{\mathbf{x}}' \cdot {\varvec{\upbeta}}} \right\} + E\left\{ {\varvec{\upvarepsilon}} \right\} = {\mathbf{x}}' \cdot {\varvec{\upbeta}} $$

(60)

$$ {\text{VAR}}\left\{ {{\mathbf{y}}'} \right\} = E\left\{ {\left[ {{\mathbf{y}}' - E\left\{ {{\mathbf{y}}'} \right\}} \right] \cdot \left[ {{\mathbf{y}}' - E\left\{ {{\mathbf{y}}'} \right\}} \right]^{\text{T}} } \right\} = E\left\{ {\left[ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}} - E\left\{ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}}} \right\}} \right] \cdot \left[ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}} - E\left\{ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}}} \right\}} \right]^{\text{T}} } \right\} = E\left\{ {{\varvec{\upvarepsilon \upvarepsilon }}^{\text{T}} } \right\} = \sigma_{\varepsilon }^{2} \cdot {\mathbf{I}} $$

(61)

Derivation of Mean and Variance of Estimated Regression Coefficients

$$ {\varvec{\hat \upbeta}} = \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{y}}' = \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot \left[ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}}} \right] $$

(62)

$$ E\left\{ {{\varvec{\hat \upbeta}}} \right\}\, = E\left\{ {\left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot \left[ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}}} \right]} \right\} = \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}' \cdot {\varvec{\upbeta}} + \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot E\left\{ {\varvec{\upvarepsilon}} \right\} = {\varvec{\upbeta}} $$

(63)

$$ VAR\left\{ {{\varvec{\hat \upbeta}}} \right\}\, = E\left\{ {\left[ {{\varvec{\hat \upbeta}} - E\left\{ {{\varvec{\hat \upbeta}}} \right\}} \right] \cdot \left[ {{\varvec{\hat \upbeta}} - E\left\{ {{\varvec{\hat \upbeta}}} \right\}} \right]^{\text{T}} } \right\} = E\left\{ {\left[ {{\varvec{\hat \upbeta}} - {\varvec{\upbeta}}} \right] \cdot \left[ {{\varvec{\hat \upbeta}} - {\varvec{\upbeta}}} \right]^{\text{T}} } \right\} = E\left\{ {\left[ {\left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{y}}' - {\varvec{\upbeta}}} \right] \cdot \left[ {\left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{y}}'} \right]^{\text{T}} - {\varvec{\upbeta}}} \right\} = E\left\{ {\left[ {\left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot \left[ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}}} \right] - {\varvec{\upbeta}}} \right] \cdot \left[ {\left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot \left[ {{\mathbf{x}}' \cdot {\varvec{\upbeta}} + {\varvec{\upvarepsilon}}} \right] - {\varvec{\upbeta}}} \right]^{\text{T}} } \right\} = \sigma_{\varepsilon }^{2} \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} $$

(64)

Derivation of the Mean and Variance of the Predicted Model Response and Its Mean

The confidence intervals and probability intervals should be reported for the predicted model response values conditional to a certain set of the input variables x′₀, the forms of which are different for linear and quadratic regression models as defined in Eqs. (65) and (66). The confidence interval corresponds to the uncertainty in the mean of the predicted model response values, and probability interval is related to the uncertainty in the prediction of the model response values.

$$ {\mathbf{x}}_{0}^{'(1)} = \left[ {\begin{array}{*{20}c} {x_{1,0} - \bar{x}_{1} } & \cdots & {x_{K,0} - \bar{x}_{K} } \\ \end{array} } \right]_{1 \times K} $$

(65)

$$ {\mathbf{x}}_{0}^{'(2)} = \left[ {\begin{array}{*{20}c} {x_{1,0} - \bar{x}_{1} } \\ \cdots \\ {x_{K,0} - \bar{x}_{K} } \\ {\left( {x_{1,0} - \bar{x}_{1} } \right) \cdot \left( {x_{1,0} - \bar{x}_{1} } \right) - \overline{{\left( {x_{1,0} - \bar{x}_{1} } \right) \cdot \left( {x_{1,0} - \bar{x}_{1} } \right)}} } \\ \cdots \\ {\left( {x_{K,0} - \bar{x}_{K} } \right) \cdot \left( {x_{K,0} - \bar{x}_{K} } \right) - \overline{{\left( {x_{K,0} - \bar{x}_{K} } \right) \cdot \left( {x_{K,0} - \bar{x}_{K} } \right)}} } \\ \end{array} } \right]_{K \cdot (K + 3)/2 \times 1}^{\text{T}} $$

(66)

The predicted model response mean and estimated model response value can be expressed as in Eqs. (67) and (68), respectively. Their corresponding expected values and variances are defined in Eqs. (69)–(72). The error is assumed to be distributed normally with zero mean and σ ² _ε variance and independent from the input variables.

$$ \hat{m}_{{y'_{0} |{\mathbf{x}}'_{0} }} = E\left\{ {y'_{0} |{\mathbf{x}}'_{0} } \right\} = {\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} $$

(67)

$$ \hat{y}'_{0} = {\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} + \hat{\varepsilon }_{0} $$

(68)

$$ E\left\{ {\hat{m}_{{y'_{0} |{\mathbf{x}}'_{0} }} } \right\} = {\mathbf{x}}'_{0} \cdot {\varvec{\upbeta}} $$

(69)

$$ E\left\{ {\hat{y}'_{0} } \right\} = {\mathbf{x}}'_{0} \cdot {\varvec{\upbeta}} $$

(70)

$$ {\text{VAR}}\left\{ {\hat{m}_{{y'_{0} |{\mathbf{x}}'_{0} }} } \right\}\, = E\left\{ {\left[ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} - E\left\{ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}}} \right\}} \right] \cdot \left[ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} - E\left\{ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}}} \right\}} \right]^{\text{T}} } \right\} = {\mathbf{x}}'_{0} \cdot {\text{VAR}}\left\{ {{\varvec{\hat \upbeta}}} \right\} \cdot \left[ {{\mathbf{x}}'_{0} } \right]^{\text{T}} = \sigma^{2} \cdot {\mathbf{x}}'_{0} \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'_{0} } \right]^{\text{T}} $$

(71)

$$ {\text{VAR}}\left\{ {\hat{y}'_{0} } \right\}\, = E\left\{ {\left[ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} + \hat{\varepsilon }_{0} - E\left\{ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} + \hat{\varepsilon }_{0} } \right\}} \right] \cdot \left[ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} + \hat{\varepsilon }_{0} - E\left\{ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} + \hat{\varepsilon }_{0} } \right\}} \right]^{\text{T}} } \right\} = E\left\{ {\left[ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} - E\left\{ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}}} \right\}} \right] \cdot \left[ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}} - E\left\{ {{\mathbf{x}}'_{0} \cdot {\varvec{\hat \upbeta}}} \right\}} \right]^{\text{T}} } \right\} + E\left\{ {\hat{\varepsilon }_{0} \cdot \left[ {\hat{\varepsilon }_{0} } \right]^{\text{T}} } \right\} = {\text{VAR}}\left\{ {\hat{m}_{{y'_{0} |{\mathbf{x}}'_{0} }} } \right\} + \sigma^{2}_\varepsilon = \sigma^{2}_\varepsilon \cdot \left( {1 + {\mathbf{x}}'_{0} \cdot \left[ {\left[ {{\mathbf{x}}'} \right]^{\text{T}} \cdot {\mathbf{x}}'} \right]^{ - 1} \cdot \left[ {{\mathbf{x}}'_{0} } \right]^{\text{T}} } \right) $$

(72)