Performance study of multi-fidelity gradient enhanced kriging

Abstract

Multi-fidelity surrogate modelling offers an efficient way to approximate computationally expensive simulations. In particular, Kriging-based surrogate models are popular for approximating deterministic data. In this work, the performance of Kriging is investigated when multi-fidelity gradient data is introduced along with multi-fidelity function data to approximate computationally expensive black-box simulations. To achieve this, the recursive CoKriging formulation is extended by incorporating multi-fidelity gradient information. This approach, denoted by Gradient-Enhanced recursive CoKriging (GECoK), is initially applied to two analytical problems. As expected, results from the analytical benchmark problems show that additional gradient information of different fidelities can significantly improve the accuracy of the Kriging model. Moreover, GECoK provides a better approximation even when the gradient information is only partially available. Further comparison between CoKriging, Gradient Enhanced Kriging, denoted by GEK, and GECoK highlights various advantages of employing single and multi-fidelity gradient data. Finally, GECoK is further applied to two real-life examples.

Appendices

Appendix A: Analytical expressions for gradient, Hessian and likelihood gradients of correlation functions

1.1 A.1 Gaussian correlation function:

Gradient of correlation function with respect to X (i.e., cross-correlation):

$$\frac{\partial\boldsymbol{\Psi}^{(i,j)}}{\partial x^{(j)}} = 2 \theta d \boldsymbol{\Psi}^{(i,j)}$$

(28)

Hessian of correlation function with respect to X (i.e., cross-correlation):

$$\begin{array}{@{}rcl@{}}\frac{\partial^{2}\boldsymbol{\Psi}^{(i,j)}}{\partial x_{u}^{(i)} \partial x_{v}^{(j)}}= \left\{\begin{array}{ll} -4\theta_u \theta_v d_u d_v \boldsymbol{\Psi}^{(i,j)} & \mathrm{if}\, u \not=\, v \\ \left[2 \theta -4 \theta^2 d^2\right]\boldsymbol{\Psi}^{(i,j)} & \mathrm{if}\, u =\, v \end{array}\right.\end{array} $$

(29)

Derivative of correlation function with respect to θ _k :

$$ \frac{\partial}{\partial \theta_{k}} \left(\psi (d_{u(v)})\right) = -10^{\theta_{u(v)}}d_{u(v)}^2log(10)exp\left(-\sum\limits_{m=1}^{k} \theta_{m} d_{m}^2 \right) $$

(30)

Derivatives of cross-correlation functions with respect to θ _k :

$$\begin{array}{@{}rcl@{}} &&\frac{\partial}{\partial \theta_{k}} \left(\frac{\partial\boldsymbol{\Psi}^{(i,j)}}{\partial x_{v}^{(j)}}\right)\nonumber\\ &&~~~~= \left\{\begin{array}{ll} 2 d_v 10^{\theta_{v}} log(10) \boldsymbol{\Psi}^{(i,j)} \left[1-10^{\theta_{k}}d_{k}^2 \right] & \text{if \(v = k\)} \\ 2 d_v 10^{\theta_{v}} log(10) \boldsymbol{\Psi}^{(i,j)} \left[-10^{\theta_{k}}d_{k}^2 \right] & \text{if \(v \not= k\)} \end{array}\right. \end{array} $$

(31)

$$\begin{array}{@{}rcl@{}} &&\frac{\partial}{\partial \theta_{k }} \left(\frac{\partial^{2}\mathbf{\Psi}^{(i,j)}}{\partial x_{u}^{(i)}\partial x_{v}^{(j)}} \right)\\ &&= \left\{\begin{array}{ll} -4 d_u d_v 10^{\theta_{u}} 10^{\theta_{v}} log(10) \mathbf{\Psi}^{(i,j)} \left[1-10^{\theta_k}d_k^2 \right] & \text{if \(u|v = k \)}\\ 4 d_u d_v d_k^210^{\theta_{u}} 10^{\theta_{v}} 10^{\theta_k} log(10) \mathbf{\Psi}^{(i,j)} & \text{otherwise}\\ \end{array}\right. \end{array} $$

(32)

$$\begin{array}{@{}rcl@{}} &&\frac{\partial}{\partial \theta_{k }} \left(\frac{\partial^{2}\boldsymbol{\Psi}^{(i,j)}}{\partial x_{u=v}^{(i)}\partial x_{u=v}^{(j)}}\right)\\ &&= \left\{\begin{array}{ll} \!\!log(10) \boldsymbol{\Psi}^{(i,j)} \!\left[2(10^{\theta}) \,+\, 4 (10^{3\theta}) d^4 \,-\, 10 (10^{2\theta}) d^2 \right] &\!\! \text{if \((u \,=\, v) \,=\, k \)} \\ \!\!-log(10) \boldsymbol{\Psi}^{(i,j)} 10^{\theta_k}d_k^2 \left[2(10^{\theta}) \,-\, 4 (10^{2\theta}) d^2 \right] & \text{\!\!if \((u\,=\,v) \!\not=\! k \)}\\ \end{array}\right.\\ \end{array} $$

(33)

1.2 A.2 Matérn $\frac {5}{2}$ correlation function:

Gradient of correlation function with respect to X (i.e., cross-correlation):

$$ \frac{\partial\boldsymbol{\Psi}^{(i,j)}}{\partial x^{(j)}} = \frac{5\theta d (\sqrt{5}a+1) exp(-\sqrt{5}a)}{3} $$

(34)

Hessian of correlation function with respect to X (i.e., cross-correlation):

$$ \frac{\partial^{2}\boldsymbol{\Psi}^{(i,j)}}{\partial x_{u}^{(i)}\partial x_{v}^{(j)}}= \left\{\begin{array}{ll}\! \frac{-25 \theta_{u} \theta_{v} d_u d_v exp(-\sqrt{5}a )}{3} & \text{if \(u \neq v\)}\\ \!\left[\frac{-25 \theta^2 d^2 + 5\theta(\sqrt{5}a+1)}{3} \right] exp(-\sqrt{5}a) & \text{if \(u = v \)} \end{array}\right. $$

(35)

Derivative of correlation function with respect to θ _k :

$$\begin{array}{@{}rcl@{}} &&\frac{\partial}{\partial \theta_{k }} (\psi_{\nu = 5/2} (d_{u(v)}))\\ &&= \frac{-(5+5\sqrt{5} a) 10^{\theta} log(10) d_{u(v)}^2 exp(-\sqrt{5}a)}{6} \end{array} $$

(36)

Derivatives of cross-correlation functions with respect to θ _k :

$$\frac{\partial}{\partial \theta_{k }} \left(\frac{\partial\boldsymbol{\Psi}^{(i, j)}}{\partial x_{v}^{(j)}}\right) \!=\! \left\{\begin{array}{ll}\!\! 10^{\theta_v} d_v C_2 \left[C_1 \!+\! \left(\frac{-25}{6}\right) 10^{\theta_{k}}d_{k}^2\right] & \text{if \(v \!\not=\! k\)} \\ \!\!10^{\theta_v} 10^{\theta_k} d_v d_k^2 \left(\frac{-25C_2}{6}\right) & \text{if \(v \!\not=\! k\)} \frac{\partial}{\partial \theta_{k }} \left(\frac{\partial\mathbf{\Psi}^{(i,j)}}{\partial x_{v}^{(j)}}\right) \,=\, \left\{\begin{array}{ll}\!\! 10^{\theta_v} d_v C_2 \left[C_1 \,+\, \left(\frac{-25}{6}\right) 10^{\theta_{k }}d_{k}^2\right] & \text{if \(v \,=\, k \)}\\ \!\!10^{\theta_v} 10^{\theta_k} d_v d_k^2 \left(\frac{-25C_2}{6}\right) & \text{if \(v \!\not=\! k\)} \end{array}\right. \end{array} \right\}$$

(37)

$$\begin{array}{@{}rcl@{}} &&\frac{\partial}{\partial \theta_{k }} \left(\frac{\partial^{2}\boldsymbol{\Psi}^{(i,j)}}{\partial x_{u}^{(i)}\partial x_{v}^{(j)}} \right) \\&&~~~~= \left\{\begin{array}{ll} \frac{-25C_2 \left(1 - \frac{\sqrt{5} 10^{\theta_{k }}d_{k}^2 }{2a} \right) 10^{\theta_u} 10^{\theta_v} d_u d_v }{3} &\text{if \(u|v = k \)}\\ \frac{C_2 25 \sqrt{5} 10^{\theta_u} 10^{\theta_v} 10^{\theta_k} d_u d_v d_{k}^2}{6a} & \mathit{otherwise}\\ \end{array}\right. \end{array} $$

(38)

$$ \frac{\partial}{\partial \theta_{k }} \left(\frac{\partial^{2}\boldsymbol{\Psi}^{(i,j)}}{\partial x_{u=v}^{(i)}\partial x_{u=v}^{(j)}}\right) = \left\{\begin{array}{ll} V_3 + V_4 & \text{if \((u = v) = k \)}\\ V_3 & \text{if \((u=v) \neq k \)} \end{array}\right. $$

(39)

where

$$ V_3 \,=\, \left[ \!\left(\frac{25\sqrt{5}}{6a}\right)(10^{\theta_{u=v}})^2(d_{u=v})^2 \,-\, \left(\frac{25}{6}\right)\!10^{\theta_{u=v}}\! \right] \!C_2 10^{\theta_{k }}d_{k}^2 $$

(40)

$$\begin{array}{@{}rcl@{}} V_4 \!&=&\! \left(\frac{-50C_2 (10^{\theta})^2d^2}{3}\right) \,+\, C_1C_2 10^{\theta}~C_1 \,=\, \left(\frac{5\sqrt{5}}{3} a + \frac{5}{3}\right)\\C_2 &=& log(10)exp\left(-\sqrt{5}a\right) \end{array} $$

(41)

Appendix B: Comparison of MLE and Least Squares Estimation (LSE) of scaling parameter (ρ)

Table 7 Comparison of NRMSE on the validation data set for different ways of ρ estimation: 1D function with n _e =4, n _c =11 and n _p =500; Peaks, inductance (L) and quality factor (Q) functions with n _e =9, n _c =30 and n _p =500 and |S ₁₁| _{m e a n} with n _e =30, n _c =60 and n _p =50. LSE(C) and LSE(L) correspond to Least Squares Estimation with constant and linear distribution of ρ, respectively

Table 8 Comparison of R ² on the validation data set for different ways of ρ estimation: 1D function with n _e =4, n _c =11 and n _p =500; Peaks, inductance ( L) and quality factor ( Q) functions with n _e =9, n _c =30 and n _p =500 and |S ₁₁| _{m e a n} with n _e =30, n _c =60 and n _p =50. LSE(C) and LSE(L) correspond to Least Squares Estimation with constant and linear distribution of ρ, respectively

Table 9 Comparison of RAAE on the validation data set for different ways of ρ estimation: 1D function with n _e =4, n _c =11 and n _p =500; Peaks, inductance ( L) and quality factor ( Q) functions with n _e =9, n _c =30 and n _p =500 and |S ₁₁| _{m e a n} with n _e =30, n _c =60 and n _p =50. LSE(C) and LSE(L) correspond to Least Squares Estimation with constant and linear distribution of ρ, respectively

Table 10 Comparison of RMAE on the validation data set for different ways of ρ estimation: 1D function with n _e =4, n _c =11 and n _p =500; Peaks, inductance ( L) and quality factor ( Q) functions with n _e =9, n _c =30 and n _p =500 and |S ₁₁| _{m e a n} with n _e =30, n _c =60 and n _p =50. LSE(C) and LSE(L) correspond to Least Squares Estimation with constant and linear distribution of ρ, respectively