Uncertainty-based multi-criteria calibration of rainfall-runoff models: a comparative study

Abstract

This study compares formal Bayesian inference to the informal generalized likelihood uncertainty estimation (GLUE) approach for uncertainty-based calibration of rainfall-runoff models in a multi-criteria context. Bayesian inference is accomplished through Markov Chain Monte Carlo (MCMC) sampling based on an auto-regressive multi-criteria likelihood formulation. Non-converged MCMC sampling is also considered as an alternative method. These methods are compared along multiple comparative measures calculated over the calibration and validation periods of two case studies. Results demonstrate that there can be considerable differences in hydrograph prediction intervals generated by formal and informal strategies for uncertainty-based multi-criteria calibration. Also, the formal approach generates definitely preferable validation period results compared to GLUE (i.e., tighter prediction intervals that show higher reliability) considering identical computational budgets. Moreover, non-converged MCMC (based on the standard Gelman–Rubin metric) performance is reasonably consistent with those given by a formal and fully-converged Bayesian approach even though fully-converged results requires significantly larger number of samples (model evaluations) for the two case studies. Therefore, research to define alternative and more practical convergence criteria for MCMC applications to computationally intensive hydrologic models may be warranted.

Appendix: review of Bayesian inference procedure

This appendix provides a summary of the Bayesian formulation used in this paper, and the details can be found in previous studies (Balin-Talamba et al. 2010; Schaefli et al. 2007). We assume the AR-based formulation as follows:

$$ Y_{i} = (Y_{i}^{sim} \left| {{\varvec{\uptheta}},{\mathbf{X}}} \right.) + \rho \varepsilon_{i - 1} + \delta_{i} $$

(3)

where Y _i and Y ^sim _i are the observed and simulated values for the model response at time step i, θ is the model parameters vector, X is the model inputs vector, ρ is the lag-one AR parameter, ɛ _i  = (Y _i  − Y ^sim _i (θ, X)) is the residual between observation and model prediction at time step i (and ɛ ₀ = 0), and δ _i is random error term:

$$ \delta_{i} \sim N(0,\sigma_{j}^{2} ) $$

(4)

with σ ² _j being the residual variance for response j, here considered unknown and should be estimated. If we consider J responses, then J parameters (representing error variance for J responses) need to be estimated in the Bayesian inference methodology. Under the assumption of multiple and statistically independent responses, the combined statistical likelihood function for multiple responses is simply the product of the individual likelihood functions:

$$ \begin{aligned} l_{multiple} & = \prod\limits_{j = 1}^{J} {l_{j} ({\varvec{\uptheta}},\rho ,\sigma_{j}^{2} ,{\mathbf{X}})} \\ \, & { = }\prod\limits_{j = 1}^{J} {\frac{1}{{\left( {\sqrt {2\pi } } \right)^{{t_{j} }} .\sigma_{j}^{{t_{j} }} }}.\exp \left( { - \frac{{\sum\limits_{i = 1}^{{t_{j} }} {\delta_{j,i}^{2} } }}{{2\sigma_{j}^{2} }}} \right)} \\ \end{aligned} $$

(5)

where δ _j,i  = ɛ _j,i  − ρɛ _j,i−1 for observation set j and time step i (note that ɛ _j,0 = 0), respectively; J is the number of observation sets, and t _j is the number of time steps for each observation set j. In order to derive the posterior distribution of parameters, a bounded uniform prior distribution is considered for θ over prior feasible range, and the prior distribution of error variance is also considered to be Jeffrey non-informative distribution as follows:

$$ p(\sigma_{j}^{2} ) \propto {1 \mathord{\left/ {\vphantom {1 {\sigma_{j}^{2} }}} \right. \kern-0pt} {\sigma_{j}^{2} }}\quad {\text{for}}\quad 0 < \sigma_{j}^{2} < \infty $$

(6)

Using such prior distributions enables us to integrate out the error variances, and the Bayesian formulation results in the joint posterior distributions from which the marginal distribution of model parameters and error variances can be estimated conditioned on the observed data Y. Alternatively, we can use MCMC sampling to directly take samples from the posterior distributions, all of which are contained in the chain. In MCMC implementations, the acceptance/rejection criterion ratio (between posterior densities of the new candidate and old current samples) is used to accept/reject the candidate to be added to the chain. In the multi-criteria Bayesian formulation, let σ ² _j,current and σ ² _j,candidate be the error variance of the current and candidate solutions, respectively, which are estimated based on the residuals after running the simulation model. Also assume the quantity $S_{j} = 0.5\sum\nolimits_{i = 1}^{{t_{j} }} {\delta_{j,i}^{2} }$, such that $S_{j,current}$ and $S_{j,candidate}$ be the values for the current and the candidate solutions, respectively. The final form of the acceptance/rejection criterion can then be shown as follows:

$$ \alpha_{{}} = \prod\limits_{i = 1}^{J} {\exp \left[ {\left( {\frac{1}{{\sigma_{j,current}^{2} }} + \frac{1}{{\sigma_{j,candidate}^{2} }}} \right)\left( {S_{j,current} - S_{j,candidate} } \right)} \right]_{{}}^{{}} {{}}^{{}} \left( {\frac{{S_{j,candidate} }}{{S_{j,current} }}} \right)^{{\frac{{t_{j} }}{2}}} } $$

(7)