A novel data-driven sparse polynomial chaos expansion for high-dimensional problems based on active subspace and sparse Bayesian learning

Abstract

Polynomial chaos expansion (PCE) has recently drawn growing attention in the community of stochastic uncertainty quantification (UQ). However, the drawback of the curse of dimensionality limits its application to complex and large-scale structures, and the PCE construction needs the complete knowledge of probability distributions of input variables, which may be impractical for real-world problems. To overcome these difficulties, this study proposes an active learning active subspace-based data-driven sparse PCE method (AL-AS-DDSPCE). First, we use the active subspace (AS) theory to reduce the dimension of the original input space, and establish the measure-consistent data-driven polynomial chaos bases in the reduced input space based on the samples of the original input random variables. Subsequently, to bypass the gradient calculation in the traditional AS method, we combine the sparse Bayesian learning with the manifold learning theory and propose an active learning AS method to obtain the subspace mapping matrix. The proposed AL-AS-DDSPCE can find the low-dimensional subspace of the original input space with the elaborate active learning algorithm, which does not require the probability distribution of input variables but is driven by the sample data of input random variables and the response data of the design samples, and can construct the PCE model efficiently and accurately. We verify the proposed method using two classical high-dimensional numerical examples, a 200-bar truss without explicit expression and one practical engineering problem. The results show that the AL-AS-DDSPCE is a good choice for solving high-dimensional UQ problems.

Appendices

Appendix 1: Brief introduction of fractional moment-based maximum entropy method

Reconstructing the PDF of a random variable Q with the finite number of moments as constraint conditions is a classical moment problem. One of the popular approaches for solving the problem is the maximum entropy method proposed by Jaynes (1957). For a continuous random variable Y, its information-theoretic entropy is defined as follows:

$$H(f(q)) = - \int_{Q} {f(q)\ln (f(q)){\text{d}}q} ,$$

(44)

where f(q) is the PDF of random variable Q. In according to maximum entropy principle, if κ accurate moments of Q are obtained, the unknown PDF, f(q), can be estimated by maximum entropy method, of which general optimization formulation is expressed as Eq. (13). When the orders of the statistical moment constrain are fractional number, Eq. (13) is named as FM-MEM.

To solve Eq. (13), its Lagrange function is constructed as follows:

$$L = - \int_{Q} {f_{{\text{e}}} (q)\ln f_{{\text{e}}} (q){\text{d}}q} + \lambda_{0} \left( {\int_{Q} {f_{{\text{e}}} (q){\text{d}}q} - 1} \right) + \sum\limits_{i = 1}^{N} {\lambda_{i} \left( {\int_{Q} {q^{{\gamma_{i} }} f_{{\text{e}}} (q){\text{d}}q} - E[Q^{{\gamma_{i} }} ]} \right)} {\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

(45)

The solution of the stationary condition of L can be analytically obtained, as shown in Eq. (14). According to the normalization axiom in probability theory, the close-form solution of λ₀ can be derived as follows:

$$\lambda_{0} = \ln \left[ {\int_{Q} {\exp ( - \sum\limits_{i = 1}^{\kappa } {\lambda_{i} } q^{{\gamma_{i} }} ){\text{d}}q} } \right].$$

(46)

It should be noted that there are two set of underdetermined variables in Eq. (13), namely, the fractional orders of the statistical moments γ = [γ₁, γ₂, …, γ_κ] and the Lagrange multipliers λ = [λ₁, λ₂, …, λ_κ]. When γ is given, λ can be solved through the statistical moment constrain conditions. Therefore, Eq. (13) is a double-loop optimization problem, where the γ-level is the outer loop, and the λ-level is the inner loop. In fact, dealing with the statistical moment constrain conditions directly is quite difficult; thus, the following equivalent form the maximum entropy method is adopted, namely, the minimization of the Kullback–Leibler divergence of the f_e(q) and the real PDF of q (Kevasan and Kapur 1992):

$$\min :\;K[f,f_{{\text{e}}} ] = \int_{Q} {f(q)\log \left[ {f(q)/f_{{\text{e}}} (q)} \right]} {\text{d}}q = - H(f(q)) - \int_{Q} {f(q)\log \left( {f_{{\text{e}}} (q)} \right)} {\text{d}}q$$

(47)

Although H is unknown, it is invariant, as shown in Eq. (44). Therefore, replacing fe(q) with Eq. (14), Eq. (47) can be rewritten as follows:

$$\min :\;\Gamma ({{\varvec{\upgamma}}},{{\varvec{\uplambda}}}) = \lambda_{0} + \sum\limits_{i = 1}^{\kappa } {\lambda_{i} } E(Q^{{\gamma_{i} }} ).$$

(48)

Finally, FM-MEM is to solve the following minimization problem:

$$\mathop {\min }\limits_{{{\varvec{\upgamma}}}} \left\{ {\mathop {\min }\limits_{{{\varvec{\uplambda}}}} \left\{ {\Gamma ({{\varvec{\upgamma}}},{{\varvec{\uplambda}}})} \right\}} \right\}.$$

(49)

To guarantee the convergence and accuracy, the simplex search method is usually used for the minimization of Eq. (49) (Zhang and Pandey 2013).

Appendix 2: The concept of Sobol’ indices

A multi-dimensional function can be expressed as the summation of a series of low-order component functions:

$$g({\mathbf{x}}) = g_{0} + \sum\limits_{{i_{1} = 1}}^{M} {g_{{i_{1} }} } (x_{{i_{1} }} ) + \sum\limits_{\begin{subarray}{l} i_{1} ,i_{2} = 1 \\ \;i_{1} < i_{2} \end{subarray} }^{M} {g_{{i_{1} ,i_{2} }} (x_{{i_{1} }} ,x_{{i_{2} }} )} + ... + g_{1,2,...,M} (x_{1} ,\;...,\;x_{M} ),$$

(50)

where x = [x₁, x₂, …, x_M] is the vector of input random variables. Equation (50) can be rewritten as a more compact form:

$$g(x_{1} ,\;...,\;x_{M} ) = \sum\limits_{{{\mathbf{u}} \subseteq \{ 1,2,...,M\} }} {g_{{\mathbf{u}}} ({\mathbf{x}}_{{\mathbf{u}}} )} ,$$

(51)

where the component functions yield

$$\begin{gathered} g_{\emptyset } = \int_{{{\mathbf{R}}^{M} }} {g({\mathbf{x}})} w({\mathbf{x}}){\text{d}}{\mathbf{x}}, \hfill \\ g_{1} = \int_{{{\mathbf{R}}^{M - 1} }} {g({\mathbf{x}})} w(x_{2} ,x_{3} ,...,x_{M} ){\text{d}}x_{2} {\text{d}}x_{3} ...{\text{d}}x_{M} - g_{\emptyset } , \hfill \\ ... \hfill \\ g_{{\mathbf{u}}} ({\mathbf{x}}_{{\mathbf{u}}} ) = \int_{{{\mathbf{R}}^{{M - ||{\mathbf{u}}||_{0} }} }} {g({\mathbf{x}}_{{\mathbf{u}}} ,{\mathbf{x}}_{{ - {\mathbf{u}}}} )} w_{{ - {\mathbf{u}}}} ({\mathbf{x}}_{{ - {\mathbf{u}}}} )d{\mathbf{x}}_{{ - {\mathbf{u}}}} - \sum\limits_{{{\mathbf{v}} \subset {\mathbf{u}}}} {g_{{\mathbf{v}}} ({\mathbf{x}}_{{\mathbf{v}}} )} , \hfill \\ \end{gathered}$$

(52)

where w(x) is the joint probability density function (PDF) of the input random variables, -u = {1, 2, …, N}\u, and ||•||₀ is the 0-norm (i.e., the number of elements of a vector). The component function, g_u(x_u), is a ||u||₀-dimensional function, representing the effect of ${\mathbf{x}}_{{\mathbf{u}}} = \left[ {x_{{i_{1} }} ,\;x_{{i_{2} }} ,\;...,\;x_{{i_{{||{\mathbf{u}}||_{0} }} }} } \right],\;1 \le i_{1} \le \ldots \le i_{{||{\mathbf{u}}||_{0} }} \le M$. The component functions in Eq. (52) are orthogonal.

The variance of g(x) can be expressed as follows:

$$D = \int_{{{\mathbf{R}}^{M} }} {\left( {g({\mathbf{x}})} \right)^{2} } w({\mathbf{x}}){\text{d}}{\mathbf{x}} - \left( {g_{\emptyset } } \right)^{2} .$$

(53)

Due to the orthogonality of the component functions, Eq. (53) can be decomposed written as follows:

$$D = \sum\limits_{{i_{1} = 1}}^{M} {D_{{i_{1} }} } + \sum\limits_{{i_{1} > i_{2} }}^{M} {D_{{i_{1} ,\;i_{2} }} } + D_{{i_{1} ,\;i_{2} ...i_{M} }} ,$$

(54)

where

$$D_{{i_{1} ,\;i_{2} ...i_{s} }} = \int_{{{\mathbf{R}}^{s} }} {\left( {g_{{i_{1} ,\;i_{2} ...i_{s} }} (x_{{i_{1} }} ,\;x_{{i_{2} }} ,\;...,\;x_{{i_{s} }} )} \right)^{2} } w(x_{{i_{1} }} ,\;x_{{i_{2} }} ,\;...,\;x_{{i_{s} }} ){\text{d}}x_{{i_{1} }} {\text{d}}x_{{i_{2} }} ...{\text{d}}x_{{i_{s} }} .$$

(55)

Thus, the partial Sobol’ index due to the cooperative effect of the input random variables $\left[ {x_{{i_{1} }} ,\;x_{{i_{2} }} ,\;...,\;x_{{i_{s} }} } \right]$ can be defined as follows:

$$S_{{i_{1} ,\;i_{2} ...i_{s} }} = \frac{{D_{{i_{1} ,\;i_{2} ...i_{s} }} }}{D}.$$

(56)

Therefore, the total Sobol’ index that represents the contribution of x_i to the variance of g(x) can be expressed as follows:

$$S_{i}^{{\text{T}}} = \sum\limits_{{{\mathbf{u}}:\;i \in {\mathbf{u}}}} {S_{{\mathbf{u}}} } .$$

(57)

Generally, the partial and total Sobol’ indices are usually named as global sensitivity index. When g(x) is replaced with its PCE model, Eqs. (1, 56 and 57) can be expressed by the PCE coefficients analytically. Interested readers can refer to Rahman (2011) and Shao et al. (2017) for details.