Accuracy evaluation of ClimGen weather generator and daily to hourly disaggregation methods in tropical conditions

Abstract

Daily and sub-daily weather data are often required for hydrological and environmental modeling. Various weather generator programs have been used to generate synthetic climate data where observed climate data are limited. In this study, a weather data generator, ClimGen, was evaluated for generating information on daily precipitation, temperature, and wind speed at four tropical watersheds located in Hawai‘i, USA. We also evaluated different daily to sub-daily weather data disaggregation methods for precipitation, air temperature, dew point temperature, and wind speed at Mākaha watershed. The hydrologic significance values of the different disaggregation methods were evaluated using Distributed Hydrology Soil Vegetation Model. MuDRain and diurnal method performed well over uniform distribution in disaggregating daily precipitation. However, the diurnal method is more consistent if accurate estimates of hourly precipitation intensities are desired. All of the air temperature disaggregation methods performed reasonably well, but goodness-of-fit statistics were slightly better for sine curve model with 2 h lag. Cosine model performed better than random model in disaggregating daily wind speed. The largest differences in annual water balance were related to wind speed followed by precipitation and dew point temperature. Simulated hourly streamflow, evapotranspiration, and groundwater recharge were less sensitive to the method of disaggregating daily air temperature. ClimGen performed well in generating the minimum and maximum temperature and wind speed. However, for precipitation, it clearly underestimated the number of extreme rainfall events with an intensity of >100 mm/day in all four locations. ClimGen was unable to replicate the distribution of observed precipitation at three locations (Honolulu, Kahului, and Hilo). ClimGen was able to reproduce the distributions of observed minimum temperature at Kahului and wind speed at Kahului and Hilo. Although the weather data generation and disaggregation methods were concentrated in a few Hawaiian watersheds, the results presented can be used to similar mountainous location settings, as well as any specific locations aimed at furthering the site-specific performance evaluation of these tested models.

Appendix

Model bias (ē) and standard error (S _e) were calculated as follows:

$$ \bar{e} = \frac{1}{N}\sum\limits_{{i = 1}}^N {({P_i} - {O_i})} $$

(14)

$$ {S_{\text{e}}} = \sqrt {{\frac{{\sum\limits_{{i = 1}}^N {{{({P_i} - {O_i})}^2}} }}{{N - 1}}}} $$

(15)

where P _i and O _i are the predicted and observed data at any given time i, and N is the total number of data points.

If the model is reliable, then S _e will be significantly smaller than the standard deviation of measured data (S _o). Thus, the ratio S _e/S _o ,which is known as relative standard error R _s, is a dimensionless measure of the improvement in the accuracy of prediction (McCuen 2003). When R _s is near zero, the model significantly improves the accuracy of prediction over the mean; however, when R _s is near 1.0, the model provides no improvement in prediction compared to the mean. Other dimensionless indices such as relative bias (R _b), relative standard error (R _s), relative difference between observed and predicted standard deviations (ΔS), and significance of difference test (Test) were calculated as follows:

$$ {R_{\text{b}}} = \frac{{\bar{e}}}{{\bar{O}}} $$

(16)

$$ \Delta {S} = \frac{{{S_{\text{p}}} - {S_{\text{o}}}}}{{{S_{\text{o}}}}} $$

(17)

$$ {\text{Test}} = \frac{{{\text{Abs}}(\bar{O} - \bar{P})}}{{2{S_{\text{e}}}}} $$

(18)

where $\bar{O}$ and $\bar{P}$ are the mean of observed and predicted weather data, respectively, and S _p is the standard deviation of predicted weather data. The closer the values of R _b, R _s, and ΔS to zero, the better the models are. The significant difference test was evaluated based on a two tailed z-test at 95% significance level. If the test values are greater than 1.0, then the difference between observed and predicted data is significant.

The index of agreement (d ₂) proposed by Willmott (1981, 1982) represents the ratio between mean square error and the potential error. The d ₂ can detect additive and proportional differences in the observed and simulated means and variances. However, d ₂ is often criticized for its over-sensitivity to extreme values due to squared differences. Legates and McCabe (1999) suggested a modified version of index of agreement (d ₁) that is less sensitive to extreme values. The index of agreement ranges between 0 and 1, where a value of 1 indicates a perfect agreement and a value of 0 indicates no agreement at all. The index of agreement (d ₂) and its modified form (d ₁) were estimated as follows:

$$ {d_2} = 1 - \frac{{\sum\limits_{{i = 1}}^N {({O_i} - {P_i}} {)^2}}}{{\sum\limits_{{i = 1}}^N {(\left| {{P_i} - \bar{O}} \right|} + {{(\left| {{O_i} - \bar{O}} \right|)}^2}}} $$

(19)

and

$$ {d_1} = 1 - \frac{{\sum\limits_{{i = 1}}^N {(\left| {{O_i} - {P_i}} \right|} {)}}}{{\sum\limits_{{i = 1}}^N {(\left| {{P_i} - \bar{O}} \right|} + {{(\left| {{O_i} - \bar{O}} \right|)}}}} $$

(20)

Table 7 Climate input scenarios to DHSVM