Gradient-enhanced kriging for high-dimensional problems

Abstract

Surrogate models provide an affordable alternative to the evaluation of expensive deterministic functions. However, the construction of accurate surrogate models with many independent variables is currently prohibitive because they require a large number of function evaluations for the desired accuracy. Gradient-enhanced kriging has the potential to reduce the number of evaluations when efficient gradient computation, such as an adjoint method, is available. However, current gradient-enhanced kriging methods do not scale well with the number of sampling points because of the rapid growth in the size of the correlation matrix, where new information is added for each sampling point in each direction of the design space. Furthermore, they do not scale well with the number of independent variables because of the increase in the number of hyperparameters that must be estimated. To address this issue, we develop a new gradient-enhanced surrogate model approach that drastically reduces the number of hyperparameters through the use of the partial least squares method to maintain accuracy. In addition, this method is able to control the size of the correlation matrix by adding only relevant points defined by the information provided by the partial least squares method. To validate our method, we compare the global accuracy of the proposed method with conventional kriging surrogate models on two analytic functions with up to 100 dimensions, as well as engineering problems of varied complexity with up to 15 dimensions. We show that the proposed method requires fewer sampling points than conventional methods to obtain the desired accuracy, or it provides more accuracy for a fixed budget of sampling points. In some cases, we get models that are over three times more accurate than previously developed surrogate models for the same computational time, and over 3200 times faster than standard gradient-enhanced kriging models for the same accuracy.

Appendices

Appendix A: Definition of the engineering cases

The analytical expressions for the engineering cases are as follows.

1.1 $\text{A.1}\;\text {P}_1,\text { P}_2, \text { and P}_3$

The responses are the deflection $\delta$, bending stress $\sigma$, and shear stress $\tau$, respectively, of a welded beam problem [13].

$$\begin{aligned} \text {P}_1:\,\delta =\frac{2.1952}{t^3b}, \end{aligned}$$

$$\begin{aligned} \text {P}_2:\,\sigma =\frac{504000}{t^2b}, \end{aligned}$$

$$\begin{aligned} \text {P}_3:\,\tau =\sqrt{\frac{\tau '^2+\tau ''^2+l\tau '\tau ''}{\sqrt{0.25\left( l^2+(h+t)^2\right) }}}, \end{aligned}$$

where

$$\begin{aligned} \tau '=\frac{6000}{\sqrt{2}hl} \text { and} \quad \tau ''=\frac{6000(14+0.5l)\sqrt{0.25\left( l^2+(h+t)^2\right) }}{2\left[ 0.707hl\left( \frac{l^2}{12}+0.25(h+t)^2\right) \right] }. \end{aligned}$$

The table below gives the ranges of the input variables.

Input variable	Range
h	[0.125, 1]
b	[0.1, 1]
l, t	[5, 10]

1.2 $\text{A.2}\;\text {P}_4$

This problem characterizes the flow of water through a borehole that is drilled from the ground surface through two aquifers [39]. The water flow rate (m$^3$/yr) is given by

$$\begin{aligned} \text {P}_4:\,y=\frac{2\pi T_u\left( H_u-H_l\right) }{\ln \left( \frac{r}{r_w}\right) \left[ 1+\frac{2LT_u}{\ln \left( \frac{r}{r_w}\right) r_w^2K_w}+\frac{T_u}{T_l}\right] }. \end{aligned}$$

The table below gives the ranges of the input variables.

Input variable	Range	Input variable	Range
$r_w$	[0.05, 0.15]	r	[100, 50000]
$T_u$	[63070, 115600]	$H_u$	[990, 1110]
$T_l$	[63.1, 116]	$H_l$	[700, 820]
L	[1120, 1680]	$K_w$	[9855, 12045]

1.3 $\text{A.3}\;\text {P}_5$

This function represents the position of a robot arm [3]:

$$\begin{aligned} \text {P}_5:\,y=\sqrt{\left( \sum \limits _{i=1}^4L_i\cos \left( \sum \limits _{j=1}^i\theta _j\right) \right) ^2+\left( \sum \limits _{i=1}^4L_i\sin \left( \sum \limits _{j=1}^i\theta _j\right) \right) ^2}. \end{aligned}$$

The table below gives the ranges of the input variables.

Input variable	Range
$L_i$	[0, 1]
$\theta _j$	$[0,2\pi ]$

1.4 $\text{A.4}\;\text {P}_6$

This is an estimate of the weight of a light aircraft wing [14]:

$$\begin{aligned} \text {P}_6:\,y=0.036S_w^{0.758}W_{fw}^{0.0035}\left( \frac{A}{\cos ^2\Lambda }\right) q^{0.006}\lambda ^{0.04}\left( \frac{100tc}{\cos \Lambda }\right) ^{-0.3}\left( N_zW_{dg}\right) ^{0.49}+S_wW_p. \end{aligned}$$

The table below gives the ranges of the input variables.

Input variable	Range	Input variable	Range
$S_w$	[150, 200]	$W_{fw}$	[220, 300]
A	[6, 10]	$\Lambda$	$[-10,10]$
q	[16, 45]	$\lambda$	[0.5, 1]
tc	[0.08, 0.18]	$N_z$	[2.5, 6]
$W_{dg}$	[1700, 2500]	$W_p$	[0.025, 0.08]

1.5 $\text{A.5}\;\text {P}_7\text { and P}_8$

These are the weight and lowest natural frequency of a torsion vibration problem [32]:

$$\begin{aligned} \text {P}_7:\,y=\sum \limits _{i=1}^3\lambda _i\pi L_i \left( \frac{d_i}{2}\right) ^2+\sum \limits _{j=1}^2\rho _j\pi T_j \left( \frac{D_j}{2}\right) ^2, \end{aligned}$$

$$\begin{aligned} \text {P}_8:\,y=\frac{\sqrt{\frac{-b-\sqrt{b^2-4c}}{2}}}{2\pi }, \end{aligned}$$

where

$$\begin{aligned} K_i=\frac{\pi G_id_i}{32L_i},\quad M_j=\frac{\rho _j \pi t_jD_j}{4g}, \end{aligned}$$

$$\begin{aligned} J_j=0.5M_j\frac{D_j}{2}, \end{aligned}$$

$$\begin{aligned} b=-\left( \frac{K_1+K2}{J_1}+\frac{K_2+K3}{J_2}\right) , \end{aligned}$$

and

$$\begin{aligned} c=\frac{K_1K_2+K_2K_3+K_3K_1}{J_1J_2}. \end{aligned}$$

The table below gives the ranges of the input variables.

Input variable	Range	Input variable	Range
$d_1$	[1.8, 2.2]	$L_1$	[9, 11]
$G_1$	[105300000, 128700000]	$\lambda _1$	[0.252, 0.308]
$d_2$	[1.638, 2.002]	$L_2$	[10.8, 13.2]
$G_2$	[5580000, 6820000]	$\lambda _2$	[0.144, 0.176]
$d_3$	[2.025, 2.475]	$L_3$	[7.2, 8.8]
$G_3$	[3510000, 4290000]	$\lambda _3$	[0.09, 0.11]
$D_1$	[10.8, 13.2]	$t_1$	[2.7, 3.3]
$\rho _1$	[0.252, 0.308]	$D_2$	[12.6, 15.4]
$t_2$	[3.6, 4.4]	$\rho _1$	[0.09, 0.11]

Appendix B: Results for the analytic cases

Table 2 Mean of the error values (upper table) and computational times (lower table) for kriging, KPLS, KPLSK, and GE-KPLSm for $m=1,\cdots ,5$ with 10 trials

Appendix C: Results for the engineering cases

Table 3 Mean of the error values (upper table) and computational times (lower table) for kriging, KPLS, KPLSK, and GE-KPLSm for $m=1,\cdots ,5$ with 10 trials