Advances in surrogate modeling for storm surge prediction: storm selection and addressing characteristics related to climate change

Abstract

This paper establishes various advancements for the application of surrogate modeling techniques for storm surge prediction utilizing an existing database of high-fidelity, synthetic storms (tropical cyclones). Kriging, also known as Gaussian process regression, is specifically chosen as the surrogate model in this study. Emphasis is first placed on the storm selection for developing the database of synthetic storms. An adaptive, sequential selection is examined here that iteratively identifies the storm (or multiple storms) that is expected to provide the greatest enhancement of the prediction accuracy when that storm is added into the already available database. Appropriate error statistics are discussed for assessing convergence of this iterative selection, and its performance is compared to the joint probability method with optimal sampling, utilizing the required number of synthetic storms to achieve the same level of accuracy as comparison metric. The impact on risk estimation is also examined. The discussion then moves to adjustments of the surrogate modeling framework to support two implementation issues that might become more relevant due to climate change considerations: future storm intensification and sea level rise (SLR). For storm intensification, the use of the surrogate model for prediction extrapolation is examined. Tuning of the surrogate model characteristics using cross-validation techniques and modification of the tuning to prioritize storms with specific characteristics are proposed, whereas an augmentation of the database with new/additional storms is also considered. With respect to SLR, the recently developed database for the US Army Corps of Engineers’ North Atlantic Comprehensive Coastal Study is exploited to demonstrate how surrogate modeling can support predictions that include SLR considerations.

Appendices

Appendix 1: Surrogate model validation

Validation of the surrogate model is established by comparing its predictions to the actual response (high-fidelity simulations of storm surge) for storm scenarios that do not belong to database X. These scenarios represent the validation set. Two common approaches are used for forming this set (Kohavi 1995): test-sample or cross-validation. The first one requires new observations, or, equivalently, splitting of the initial database to a single training set and a single validation set. The second approach repeats comparisons of the accuracy over different divisions of the entire database to training set and validation set and has the benefit that it requires no new observations. Perhaps the most common implementation of the latter is the leave-one-out cross-validation (LOOCV): Each of the observations from the database is sequentially removed, then the remaining training points are used to predict the output for it, and the error between the predicted and real responses is evaluated. A more general CV approach is k-fold CV, established by partitioning the entire observation set into k equal-size subsets and calculating errors similarly as in the LOOCV, but removing each time the entire subset, rather than a single observation.

As error statistics popular choices are the coefficient of determination, the correlation coefficient and the square root of the mean squared error. For output y_j and the LOOCV approach these are given by

$$R_{j}^{2} = 1 - \frac{{\sum\nolimits_{h = 1}^{n} {\left( {y_{j} ({\mathbf{x}}^{h} ) - \hat{y}_{j} ({\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} )} \right)^{2} } }}{{\sum\nolimits_{h = 1}^{n} {\left( {y_{j} ({\mathbf{x}}^{h} ) - \bar{y}_{j} } \right)^{2} } }}; \, \bar{y}_{j} = \frac{1}{n}\sum\limits_{h = 1}^{n} {y_{j} ({\mathbf{x}}^{h} )}$$

(18)

$$ {{cc}}_{j} = \frac{{\sum\nolimits_{h = 1}^{n} {\left( {y_{j} ({\mathbf{x}}^{h} ) - \bar{y}_{j} } \right)\left( {\hat{y}_{j} ({\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} ) - \bar{\hat{y}}_{i} } \right)} }}{{\sqrt {\sum\nolimits_{h = 1}^{n} {\left( {y_{j} ({\mathbf{x}}^{h} ) - \bar{y}_{j} } \right)^{2} } \sum\nolimits_{h = 1}^{n} {\left( {\hat{y}_{j} \left( {{\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} } \right) - \bar{\hat{y}}_{j} ({\mathbf{X}}_{ - h} )} \right)^{2} } } }}; \, \bar{\hat{y}}_{j} ({\mathbf{X}}_{ - h} ) = \frac{1}{n}\sum\limits_{h = 1}^{n} {\hat{y}_{j} \left( {{\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} } \right)}$$

(19)

$$ {{rmse}}_{j} = \sqrt {\frac{1}{n}\sum\limits_{h = 1}^{n} {\left( {y_{j} ({\mathbf{x}}^{h} ) - \hat{y}_{j} ({\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} )} \right)^{2} } } , { }$$

(20)

where $\hat{y}_{j} ({\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} )$ represents the surrogate model predictions for storm x^h established by using the entire database excluding that storm. Using the closed form of Eq. (7) the statistics in Eqs. (18)–(20) can be calculated without explicitly evaluating $\hat{y}_{j} ({\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} )$, rather simply substituting LOO predictions for entire output vector by

$${\hat{\mathbf{y}}}\left( {{\mathbf{x}}^{h} |{\mathbf{X}}_{ - h} } \right) = {\mathbf{P}}\left( {{\mathbf{z}}({\mathbf{x}}^{h} ) + [{\mathbf{e}}]_{h}^{\text{T}} } \right),$$

(21)

where [e]_h is the vector corresponding to the hth row of matrix with elements e_hi, i.e., to the LOO errors associated with the latent outputs for storm x^h. For the test-sample approach the set $\{ {\mathbf{x}}^{h} ;h = 1, \ldots ,n\}$ needs to be replaced with the training set of new observations $\{ {\mathbf{x}}_{n}^{h} ;h = 1, \ldots ,n_{n} \}$ in these equations, whereas the entire database X (rather than X_–h) is used for the surrogate model predictions. It should be pointed out that the aforementioned statistics evaluate the average accuracy over the examined domain X. An alternative approach would have been to look at the maximum error over X which examines the worst-case prediction performance.

The accuracy across all outputs is evaluated by calculating the average error statistics over all outputs.

$$ \bar{R}^{\textit{2}} = \frac{\text{1}}{{n_{y} }}\sum\limits_{j = \textit{1}}^{{n_{y} }} {R_{j}^{\textit{2}} } ;\quad \, \overline{\textit{cc}} = \frac{\text{1}}{{n_{y} }}\sum\limits_{j = \textit{1}}^{{n_{y} }} {{\textit{cc}}_{j} } ;\quad \overline{\textit{rmse}} = \frac{\text{1}}{{n_{y} }}\sum\limits_{j = \textit{1}}^{{n_{y} }} {{\textit{rmse}}_{j} } .$$

(22)

Values of $\bar{R}^{2}$ and $\overline{{cc}}$ close to 1 and values of $\overline{{rmse}}$ close to zero indicate better accuracy. It should be also pointed out that $\overline{{rmse}}$ does not correspond to normalized statistics; in other words, its value depends on the magnitude of the response.

Appendix 2: Calculation of gradients for LOOCV objective function

This appendix discusses the estimation of the gradient of the objective function of Eq. (6) utilized in the LOOCV tuning of the hyper-parameters. The kth component of the gradient vector, the partial derivative with respect to hyper-parameter s_k, is

$$\frac{{\partial H_{m} ({\mathbf{s}})}}{{\partial s_{k} }} = \frac{2}{{m_{c} }}\sum\limits_{i = 1}^{{m_{c} }} {\gamma_{i} } \left( {\frac{1}{n}\sum\limits_{h = 1}^{n} {e_{hi} \frac{{\partial e_{hi} }}{{\partial s_{k} }}} } \right)$$

(23)

with partial derivative for LOO error given by

$$\frac{{\partial e_{hi} }}{{\partial s_{k} }} = \frac{{\partial \left( {g_{hi} /c_{hh} } \right)}}{{\partial s_{k} }} = \left( {c_{hh} \frac{{\partial g_{hi} }}{{\partial s_{k} }} - g_{hi} \frac{{\partial c_{hh} }}{{\partial s_{k} }}} \right)/(c_{hh} )^{2} ,$$

(24)

where

$$\begin{aligned} \frac{{\partial g_{hi} }}{{\partial s_{k} }} & = - \left[ {{\mathbf{R}}({\mathbf{X}})^{ - 1} \cdot \frac{{\partial {\mathbf{R}}({\mathbf{X}})}}{{\partial s_{k} }} \cdot {\mathbf{R}}({\mathbf{X}})^{ - 1} ({\mathbf{Z}} - {\mathbf{F}}({\mathbf{X}}){\varvec{\upbeta}}^{*} ) + {\mathbf{R}}({\mathbf{X}})^{ - 1} {\mathbf{F}}({\mathbf{X}})\frac{{\partial {\varvec{\upbeta}}^{*} }}{{\partial s_{k} }}} \right]_{hi} \\ \frac{{\partial c_{hh} }}{{\partial s_{k} }} & = - {\mathbf{R}}({\mathbf{X}})^{ - 1} \left[ {\frac{{\partial {\mathbf{R}}({\mathbf{X}})}}{{\partial s_{k} }}} \right]_{hh} {\mathbf{R}}({\mathbf{X}})^{ - 1} , \\ \end{aligned}$$

(25)

in which $\frac{{\partial {\mathbf{R}}({\mathbf{X}})}}{{\partial s_{k} }}$ and $\frac{{\partial {\varvec{\upbeta}}^{*} }}{{\partial s_{k} }}$ are matrices of element-wise derivatives with the latter given by

$$\frac{{\partial {\varvec{\upbeta}}^{*} }}{{\partial s_{k} }} = \left( {{\mathbf{Q}}({\mathbf{X}})^{\text{T}} {\mathbf{F}}({\mathbf{X}})} \right)^{ - 1} {\mathbf{Q}}({\mathbf{X}})^{\text{T}} \cdot \frac{{\partial {\mathbf{R}}({\mathbf{X}})}}{{\partial s_{k} }} \cdot \left( {{\mathbf{Q}}({\mathbf{X}})\left( {{\mathbf{Q}}({\mathbf{X}})^{\text{T}} {\mathbf{F}}({\mathbf{X}})} \right)^{ - 1} {\mathbf{F}}({\mathbf{X}})^{\text{T}} - {\mathbf{I}}} \right){\mathbf{R}}({\mathbf{X}})^{ - 1} {\mathbf{Z}} \,$$

(26)

with ${\mathbf{Q}}({\mathbf{X}})^{\text{T}} = {\mathbf{F}}({\mathbf{X}})^{\text{T}} {\mathbf{R}}({\mathbf{X}})^{ - 1}$.

Appendix 3: Optimization for storm selection

This appendix discusses a numerical optimization scheme for the storm selection. First, select the number of candidate samples n_c for the stochastic search, the percentage reductions a_r, a_c for determining the retained experiments, and the number of samples for the Monte Carlo integration (MCI) N_s. Perform then the following steps, also shown in Fig. 11 for the same example considered in Fig. 1.

Fig. 11

Demonstration of the different steps for the numerical implementation of the adaptive DoE for the same two-dimensional example shown in Fig. 1. In all the curves the contours correspond to the normalized variance. a Initial candidate experiments; b experiments retained after a 50% ranking based on [a_r = 0.5]; c experiments retained after 50% clustering [a_c = 0.5]; d final experiment obtained after minimization of IMSE

• (1) [MCI samples]. Generate N_s {x^q; q = 1,…, N_s} samples representing distribution φ(x), and determine the corresponding weights w_q(x^q). For the common implementation that samples are directly from distribution φ(x) the weights are w_q(x^q) = 1.

• (2) [Candidate experiment generation]. Generate n_c samples for x with uniform distribution in domain D, forming set $\{ {\mathbf{x}}_{new}^{c} ;c = 1, \ldots ,n_{c} \}$. Skip this step if candidate experiments are already provided (Part a of Fig. 11).

• (3) [Ranking of experiments]. Evaluate $\{ \sigma_{n}^{2} ({\mathbf{x}}_{new}^{c} |{\mathbf{X}});c = 1, \ldots ,n_{c} \}$ and order candidate samples using a descending order for $\sigma_{n}^{2} ({\mathbf{x}}_{new}^{c} |{\mathbf{X}})$. Retain only the a_rn_c candidate experiments that correspond to the highest values of $\{ \sigma_{n}^{2} ({\mathbf{x}}_{new}^{c} |{\mathbf{X}});c = 1, \ldots ,n_{c} \}$ (Part b of Fig. 11). Skip this step if a_r = 1.

• (4) [Clustering of solutions]. Cluster retained experiments from Step 3 into a_ca_rn_c clusters, using for example K-means cluster (Hartigan and Wong 1979), and keep only one experiment per cluster, the one corresponding to the largest value of $\sigma_{n}^{2} ({\mathbf{x}}_{new}^{c} |{\mathbf{X}})$ (Part c of Fig. 11). Skip this step if a_c = 1.

• (5) [Calculation of IMSE]. For each retained candidate experiment after Step 4, $\{ {\mathbf{x}}_{new}^{c} ;c = 1, \ldots ,a_{r} a_{c} n_{c} \}$, calculate $\{ \sigma_{n}^{2} ({\mathbf{x}}^{q} |{\mathbf{X}},{\mathbf{x}}_{new}^{c} );q = 1, \ldots ,N_{q} \}$ using Eqs. (12) and (13). Approximate IMSE through MCI as

  $$IMSE({\mathbf{x}}_{new}^{c} ) = \frac{1}{{N_{s} }}\sum\limits_{q = 1}^{{N_{s} }} {w_{q} ({\mathbf{x}}^{q} )\sigma_{n}^{2} ({\mathbf{x}}^{q} |{\mathbf{X}},{\mathbf{x}}_{new}^{c} )} .$$

  (27)

• (6) [Final selection]. Select as new experiment the one that provides the minimum value for $\{ IMSE({\mathbf{x}}_{new}^{c} );c = 1, \ldots ,a_{r} a_{c} n_{c} \}$ (Part d of Fig. 11).

The most computationally demanding step of this process is Step 5, which requires MCI for each of the candidate experiments examined. Steps 3 and 4 have been introduced to reduce this burden. Step 3 removes candidate experiments belonging to domains where the surrogate model accuracy is already high. For such experiments, it is anticipated that their IMSE will not correspond to a minimum over D, as addition of experiments in domains of already adequate accuracy [corresponding to lower values of $\sigma_{n}^{2} ({\mathbf{x}}|{\mathbf{X}})$] is anticipated to provide small benefits. Step 4 keeps only one candidate experiment among experiments that are close to one another. All such experiments are expected to provide similar improvement of IMSE, and therefore, examining all of them as candidate solutions is redundant. Only the experiment with lowest accuracy is retained in each cluster. Removal of experiments in Steps 3 and 4 provides an a_ca_r-fold reduction in computational burden without compromising the quality of the identified solution of the optimization, provided that values of a_r and a_c are not too small. These values should be chosen in range [0.7–0.3].