Regularized Bayesian estimation for GEV-B-splines model

Abstract

Large observed datasets are not stationary and/or depend on covariates, especially, in the case of extreme hydrometeorological variables. This causes the difficulty in estimation, using classical hydrological frequency analysis. A number of non-stationary models have been developed using linear or quadratic polynomial functions or B-splines functions to estimate the relationship between parameters and covariates. In this article, we propose regularised generalized extreme value model with B-splines (GEV-B-splines models) in a Bayesian framework to estimate quantiles. Regularisation is based on penalty and aims to favour parsimonious model especially in the case of large dimension space. Penalties are introduced in a Bayesian framework and the corresponding priors are detailed. Five penalties are considered and the corresponding priors are developed for comparison purpose as: Least absolute shrinkage and selection (Lasso and Ridge) and smoothing clipped absolute deviations (SCAD) methods (SCAD1, SCAD2 and SCAD3). Markov chain Monte Carlo (MCMC) algorithms have been developed for each model to estimate quantiles and their posterior distributions. Those approaches are tested and illustrated using simulated data with different sample sizes. A first simulation was made on polynomial B-splines functions in order to choose the most efficient model in terms of relative mean biais (RMB) and the relative mean-error (RME) criteria. A second simulation was performed with the SCAD1 penalty for sinusoidal dependence to illustrate the flexibility of the proposed approach. Results show clearly that the regularized approaches leads to a significant reduction of the bias and the mean square error, especially for small sample sizes (n < 100). A case study has been considered to model annual peak flows at Fort-Kent catchment with the total annual precipitations as covariates. The conditional quantile curves were given for the regularized and the maximum likelihood methods.

Appendices

Appendix 1: Bayesian Lasso GEV B-splines

The expression of Lasso penalty is:

$$p_{l} \left( \beta \right) = {\mathop \sum \limits_{j = 1}^{m + l + 1}} |\beta_{j} |\;{\text{With}}\;{\text{L}}_{1}\!-\!{\text{norm}}$$

(23)

1.1 Proposed model (GEV-Lasso)

$$\pi (\beta ) \propto \pi \left( {\beta_{j} |\tau^{2} } \right)*\pi \left( {\tau^{2} |\lambda } \right)*\pi \left( {\lambda^{2} } \right)$$

(24)

$$f\left( {\theta |y} \right) \propto \frac{1}{\sigma }\left\{ {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right\}^{{ - \frac{1}{\kappa } - 1}} *{ \exp }\left( { - \left( {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right)^{{ - \frac{1}{\kappa }}} } \right)$$

(25)

$$*\mathop \prod \limits_{j = 1}^{d} \exp \left( {\frac{{ - \beta_{j}^{2} }}{{2\tau^{2} }}} \right)*\exp \left( {\frac{{\lambda^{2} }}{2}} \right)*\left( {\frac{{h^{2} }}{{{{\Gamma }}\left( \lambda \right)}}\lambda^{2z - 2} \exp \left( { - h\lambda^{2} } \right)} \right)*\frac{1}{\sigma }*\left( {\left( {\frac{\kappa - a}{b - a}} \right)^{\alpha - 1} \left( {1 - \left( {\frac{\kappa - a}{b - a}} \right)} \right)^{\beta - 1} \frac{{{{\Gamma }}(u + v)}}{{{{\Gamma }}(u){{\Gamma }}(v)}}} \right)$$

Appendix 2: Ridge GEV-B-splines model

2.1 Penalty

$$P_{l} \left( \beta \right) = \mathop \sum \limits_{j = 1}^{m + l + 1} \beta_{j}^{2} = \lambda \mathop \sum \limits_{j = 1}^{m + l + 1} \beta_{j}^{2}$$

(26)

2.2 Proposed model (GEV-Ridge)

$$\pi (\beta ) \propto \mathop \prod \limits_{j = 1}^{d} exp\left( {\frac{{ - \beta^{2} }}{{2\tau^{2} }}} \right)*\frac{{2S^{\upsilon } }}{\upsilon - 2}*\tau^{ - (\upsilon + 2)} { \exp }\left( { - S^{2} \tau^{ - 2} } \right)$$

(27)

$$f\left( {\theta |y} \right) \propto \frac{1}{\sigma }\left\{ {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right\}^{{ - \frac{1}{\kappa } - 1}} *exp\left( { - \left( {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right)^{{ - \frac{1}{\kappa }}} } \right)$$

$$* \mathop \prod \limits_{j = 1}^{d} exp\left( {\frac{{ - \beta^{2} }}{{2\tau^{2} }}} \right)*\frac{{2S^{\upsilon } }}{\upsilon - 2}*\tau^{{ - \left( {\upsilon + 2} \right)}} { \exp }\left( { - S^{2} \tau^{ - 2} } \right)*\frac{1}{\sigma }* \left( {\left( {\frac{\kappa - a}{b - a}} \right)^{\alpha - 1} \begin{array}{*{20}l} { } \\ {\left( {1 - \left( {\frac{\kappa - a}{b - a}} \right)} \right)} \\ \end{array}^{\beta - 1} \frac{{{{\Gamma }}(u + v)}}{{{{\Gamma }}(u){{\Gamma }}(v)}}} \right)$$

(28)

Appendix 3: SCAD GEV-B-splines

3.1 SCAD1

3.1.1 Penalty

$$P_{\lambda } \left( {\left| \beta \right|} \right) = \lambda \left\{ {\begin{array}{lll} \!\!{\left| \beta \right|} & \quad{if\; \left| \beta \right|~ \le \lambda } \\ \!\!\!{- \frac{{\left( {\beta ^{2} - 2a~\lambda \left| \beta \right| + \lambda ^{2} } \right)}}{{2\left( {a - 1} \right)\lambda }}~} &\quad {\forall ~\lambda < \left| \beta \right| \le a\lambda } \\ \!\!{\frac{1}{2}~\left( {a + 1} \right)\lambda} \\ \end{array} } \right.$$

(29)

3.1.2 Proposed model (GEV-SCAD1)

$$f\left( {\theta |y} \right) \propto \frac{1}{\sigma }\left\{ {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right\}^{{ - \frac{1}{\kappa } - 1}} *exp\left( { - \left( {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right)^{{ - \frac{1}{\kappa }}} } \right)*\frac{1}{\sigma }*\left( {\left( {\frac{\kappa - a}{b - a}} \right)^{\alpha - 1} \left( {1 - \left( {\frac{\kappa - a}{b - a}} \right)} \right)^{\beta - 1} \frac{{{{\Gamma }}(u + v)}}{{{{\Gamma }}(u){{\Gamma }}(v)}}} \right)*\frac{1}{{(a\lambda )^{2} }}*\mathop \prod \limits_{j = 1}^{d} exp\left( {\frac{{ - \beta^{2} }}{{2(a\lambda )^{2} }}} \right)$$

(30)

3.2 SCAD2

3.2.1 Penalty

$$P_{\lambda } \left( \beta \right) \approx \mathop \sum \limits_{j = 1}^{d} P_{\lambda } \left( {\beta_{j} } \right) = P_{\lambda } \left( {|\beta_{j}^{0} |} \right) + \mathop \sum \limits_{j = 1}^{d} P_{\lambda }^{'} \left( {\beta_{j}^{{0^{2} }} } \right)\left( {\beta_{j}^{2} - \beta_{j}^{{(0)^{2} }} } \right)$$

(31)

3.2.2 Proposed model (GEV-SCAD2)

$$f\left( {\theta |y} \right) \propto \frac{1}{\sigma }\left\{ {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right\}^{{ - \frac{1}{\kappa } - 1}} *exp\left( { - \left( {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right)^{{ - \frac{1}{\kappa }}} } \right)*\frac{1}{\sigma }*\left( {\left( {\frac{\kappa - a}{b - a}} \right)^{\alpha - 1} \left( {1 - \left( {\frac{\kappa - a}{b - a}} \right)} \right)^{\beta - 1} \frac{{{{\Gamma }}(u + v)}}{{{{\Gamma }}(u){{\Gamma }}(v)}}} \right)*2P_{\lambda }^{'} \left( {\beta_{j}^{{0^{2} }} } \right)*\mathop \prod \limits_{j = 1}^{d} exp\left( { - \beta^{2} P_{\lambda }^{'} \left( {\beta_{j}^{{0^{2} }} } \right)} \right)$$

(32)

3.3 SCAD3

3.3.1 Penalty

$$P_{\lambda } \left( \beta \right) \approx \mathop \sum \limits_{j = 1}^{d} P_{\lambda } \left( {\beta_{j} } \right) = P_{\lambda } \left( {|\beta_{j}^{(0)} |} \right) + \mathop \sum \limits_{j = 1}^{d} P_{\lambda }^{'} \left( {\beta_{j}^{(0)} } \right)\left( {\beta_{j} - \beta_{j}^{(0)} } \right)$$

(33)

3.3.2 Proposed model (GEV-SCAD3)

$$f\left( {\theta |y} \right) \propto \frac{1}{\sigma }\left\{ {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right\}^{{ - \frac{1}{\kappa } - 1}} *exp\left( { - \left( {1 + \kappa \left( {\frac{{y - \mathop \sum \nolimits_{j} \left( {1 B} \right)*\left( {\begin{array}{*{20}l} {\beta_{0} } \\ \beta \\ \end{array} } \right)}}{\sigma }} \right)} \right)^{{ - \frac{1}{\kappa }}} } \right)*\frac{1}{\sigma }*\left( {\left( {\frac{\kappa - a}{b - a}} \right)^{\alpha - 1} \left( {1 - \left( {\frac{\kappa - a}{b - a}} \right)} \right)^{\beta - 1} \frac{{{{\Gamma }}(u + v)}}{{{{\Gamma }}(u){{\Gamma }}(v)}}} \right)*2P_{\lambda }^{'} \left( {\beta_{j}^{(0)} } \right)*\mathop \prod \limits_{j = 1}^{d} exp\left( { - \beta^{2} 2P_{\lambda }^{'} \left( {\beta_{j}^{(0)} } \right)} \right)$$

(34)