An adaptive sparse polynomial dimensional decomposition based on Bayesian compressive sensing and cross-entropy

Abstract

The polynomial dimensional decomposition (PDD) is a powerful tool for uncertainty quantification. When dealing with complex or high-dimensional problems, the computational cost of the PDD is key to its application. This study proposes a novel sparse PDD method integrated with the Bayesian LASSO (least absolute shrinkage and selection operator) method and an adaptive polynomial basis updating method. Firstly, we construct the sparse PDD metamodel using an analytical Bayesian LASSO method, which is developed based on the framework of sparse Bayesian learning. The analytical method employs an iteration formula to calculate the maximum posterior estimation instead of the time-consuming Markov chain Monte Carlo (MCMC) sampling, and therefore can be used for refining the PDD model repeatedly. Secondly, to improve the performance of the analytical Bayesian LASSO, this paper proposes a cross-entropy-based method for updating the sparse PDD model, which allows the sequential augmentation of the polynomial bases and adaptively determines the polynomial degree and the maximum order of the component functions. The cross-entropy-based method can guarantee a small number of polynomial bases when refining the sparse model. Accordingly, the computational accuracy can be improved with the same sample size. We verify the proposed method using three numerical benchmark examples, and apply it to solve one complex practical engineering problem. The results show that the proposed sparse PDD method is a good choice for uncertainty quantification.

Appendices

Appendix 1

A one-dimensional polynomial chaos basis of θ is defined through the following orthogonality condition:

$$\left\langle {\phi_{i} (\theta ),\phi_{j} (\theta )} \right\rangle = \int {\phi_{i} (\theta )\phi_{j} (\theta )f{(}\theta {\text{)d}}\theta } = \delta_{ij} ,$$

(47)

where δ_ij is the Kronecker symbol, and f(θ) is the PDF of the random variable θ. For arbitrary f(θ), the orthogonal polynomial basis, ϕ_i, can be derived by Stieltjes procedure (Wan and Karniadakis 2006) with the following recurrence relation

$$\sqrt {\beta_{n + 1} } \phi_{n + 1} (\theta ) = (\theta - \alpha_{n} )\phi_{n} (\theta ) - \sqrt {\beta_{n} } \phi_{n - 1} (\theta )\quad \quad n = 1,\;2,\;\ldots$$

(48)

where α_n and β_n are given by the Christoffel–Darboux formulae as follows:

$$\begin{gathered} \alpha_{n} = \frac{{\left\langle {\zeta \hat{\phi }_{n} ,\;\hat{\phi }_{n} } \right\rangle }}{{\left\langle {\hat{\phi }_{n} ,\;\hat{\phi }_{n} } \right\rangle }} \hfill \\ \beta_{n} = \frac{{\left\langle {\hat{\phi }_{n} ,\;\hat{\phi }_{n} } \right\rangle }}{{\left\langle {\hat{\phi }_{n - 1} ,\;\hat{\phi }_{n - 1} } \right\rangle }} \hfill \\ \phi_{n} = \frac{{\hat{\phi }_{n} }}{{\sqrt {\left\langle {\hat{\phi }_{n} ,\;\hat{\phi }_{n} } \right\rangle } }} \hfill \\ \end{gathered}$$

(49)

With Eqs. (48) and (49), polynomial chaos bases can be derived for the random variable with arbitrary PDF. For ease of use, the corresponding polynomial chaos bases for θ with the common distributions are shown in Table 1.

Appendix 2

Herein, the detailed derivation of Eq. (26) is provided. Because both p(g_s|c, ω_MAP) and p(c|ω_MAP) are normal distributions, Eq. (25) can be rewritten as the following expression, dropping off the terms that are not related to c

$$\begin{gathered} p\left( {{\mathbf{c}}{|}\;{{\varvec{\upomega}}}_{{{\text{MAP}}}} ,\;{\mathbf{g}}_{{\text{s}}} } \right) \propto p\left( {{\mathbf{g}}_{{\text{s}}} {|}\;{\mathbf{c}},\;{{\varvec{\upomega}}}_{{{\text{MAP}}}} } \right)p\left( {{\mathbf{c}}{|}\;{{\varvec{\upomega}}}_{{{\text{MAP}}}} } \right) \hfill \\ \quad \quad \quad \quad \quad \;\;\; \propto \exp \left( { - \frac{1}{{2\sigma_{{{\text{MAP}}}}^{2} }}\left( {{\mathbf{g}}_{{\text{s}}} - {\mathbf{H}}^{{\text{T}}} {\mathbf{c}}} \right)^{{\text{T}}} \left( {{\mathbf{g}}_{{\text{s}}} - {\mathbf{H}}^{{\text{T}}} {\mathbf{c}}} \right) - \frac{1}{2}{\mathbf{c}}^{{\text{T}}} {\mathbf{Ac}}} \right) \hfill \\ \quad \quad \quad \quad \quad \;\;\; \propto \exp \left( { - \frac{1}{2}\left( {{\mathbf{c}}^{{\text{T}}} \left( {\frac{{{\mathbf{HH}}^{{\text{T}}} }}{{\sigma_{{{\text{MAP}}}}^{2} }} + {\mathbf{A}}} \right){\mathbf{c}} - \frac{2}{{\sigma_{{{\text{MAP}}}}^{2} }}{\mathbf{g}}_{{\text{s}}}^{{\text{T}}} {\mathbf{H}}^{{\text{T}}} {\mathbf{c}}} \right)} \right). \hfill \\ \end{gathered}$$

(50)

Eq. (50) is nothing but the kernel of a Gaussian distribution. The corresponding covariance matrix is the inverse of the quadratic term coefficient, and the mean value vector can be easily obtained by the ratio of the linear term coefficient to the quadratic term coefficient; that is,

$${\mathbf{c}}{|}\;{{\varvec{\upomega}}}_{{{\text{MAP}}}} ,\;{\mathbf{g}}_{{\text{s}}} \sim {\text{N(}}{{\varvec{\upmu}}},\;{{\varvec{\Sigma}}}),$$

(51)

where

$$\begin{gathered} {{\varvec{\upmu}}} = 1/\sigma_{{^{{{\text{MAP}}}} }}^{2} {\mathbf{\Sigma Hg}} \hfill \\ {{\varvec{\Sigma}}} = (1/\sigma_{{^{{{\text{MAP}}}} }}^{2} {\mathbf{HH}}^{{\text{T}}} + {\mathbf{A}})^{ - 1} . \hfill \\ \end{gathered}$$

(52)

Appendix 3

The derivation of Eq. (28) is based on the results of Appendix 2. Substituting Eqs. (18) and (51) into Eq. (24), we have

$$\begin{gathered} p\left( {g_{{\text{s}}}^{*} {|}{\mathbf{g}}_{{\text{s}}} } \right) \approx \int {p\left( {g_{{\text{s}}}^{*} {|}{\mathbf{c}},\;{{\varvec{\upomega}}}_{{{\text{MAP}}}} ,\;{\mathbf{g}}_{{\text{s}}} } \right)p\left( {{\mathbf{c}}{|}\;{{\varvec{\upomega}}}_{{{\text{MAP}}}} ,\;{\mathbf{g}}_{{\text{s}}} } \right)} {\text{d}}{\mathbf{c}} \hfill \\ \quad \quad \quad \;\; \propto \int {\exp \left( { - \frac{1}{{2\sigma_{{{\text{MAP}}}}^{2} }}\left( {g_{{\text{s}}}^{*} - {{\varvec{\Phi}}}^{{\text{T}}} \left( {{\mathbf{x}}^{*} } \right){\mathbf{c}}} \right)^{2} } \right)\exp \left( { - \frac{1}{2}\left( {{\mathbf{c}} - {{\varvec{\upmu}}}} \right)^{{\text{T}}} {{\varvec{\Sigma}}}^{ - 1} \left( {{\mathbf{c}} - {{\varvec{\upmu}}}} \right)} \right)} {\text{d}}{\mathbf{c}}. \hfill \\ \end{gathered}$$

(53)

After some basic algebraic operations and dropping off the terms that are not related to $g_{{\text{s}}}^{*}$ and c, Eq. (53) can be further written as

$$p\left( {g_{{\text{s}}}^{*} {|}{\mathbf{g}}_{{\text{s}}} } \right)\; \propto \exp \left( { - \frac{1}{{2\sigma_{{{\text{MAP}}}}^{2} }}\left( {g_{{\text{s}}}^{*} } \right)^{2} } \right)\int {\exp \left( { - \frac{1}{2}\left( {{\mathbf{c}}^{{\text{T}}} {\mathbf{\Omega c}} - 2{\mathbf{c}}^{{\text{T}}} {{\varvec{\Gamma}}}} \right)} \right)} {\text{d}}{\mathbf{c}},$$

(54)

where

$$\begin{gathered} {{\varvec{\Omega}}} = \left( {\frac{{{{\varvec{\Phi}}}\left( {{\mathbf{x}}^{*} } \right){{\varvec{\Phi}}}^{{\text{T}}} \left( {{\mathbf{x}}^{*} } \right)}}{{\sigma_{{{\text{MAP}}}}^{2} }} + {{\varvec{\Sigma}}}^{ - 1} } \right) \hfill \\ {{\varvec{\Gamma}}} = \frac{{g_{{\text{s}}}^{*} {{\varvec{\Phi}}}\left( {{\mathbf{x}}^{*} } \right)}}{{\sigma_{{{\text{MAP}}}}^{2} }} + {{\varvec{\Sigma}}}^{ - 1} {{\varvec{\upmu}}}. \hfill \\ \end{gathered}$$

(55)

Thus, it can be seen that a Gaussian kernel can be constructed by the second term of Eq. (54); that is,

$$p\left( {g_{{\text{s}}}^{*} {|}{\mathbf{g}}_{{\text{s}}} } \right)\; \propto \exp \left( { - \frac{{\left( {g_{{\text{s}}}^{*} } \right)^{2} }}{{2\sigma_{{{\text{MAP}}}}^{2} }} + \frac{1}{2}{{\varvec{\Gamma}}}^{{\text{T}}} {{\varvec{\Omega}}}^{ - 1} {{\varvec{\Gamma}}}} \right)\int {\exp \left( { - \frac{{({\mathbf{c}} - {{\varvec{\Omega}}}^{ - 1} {{\varvec{\Gamma}}})^{{\text{T}}} {{\varvec{\Omega}}}({\mathbf{c}} - {{\varvec{\Omega}}}^{ - 1} {{\varvec{\Gamma}}})}}{2}} \right)} {\text{d}}{\mathbf{c}}.$$

(56)

The integration in Eq. (56) is constant because of the Gaussian kernel; therefore, we have

$$p\left( {g_{{\text{s}}}^{*} {|}{\mathbf{g}}_{{\text{s}}} } \right)\; \propto \exp \left( { - \frac{1}{{2\sigma_{{{\text{MAP}}}}^{2} }}\left( {g_{{\text{s}}}^{*} } \right)^{2} + \frac{1}{2}{{\varvec{\Gamma}}}^{{\text{T}}} {{\varvec{\Omega}}}^{ - 1} {{\varvec{\Gamma}}}} \right).$$

(57)

Combining Eq. (55) with (57), one can confirm that Eq. (57) is also a Gaussian kernel; therefore, it can be written as follows:

$$p\left( {g_{{\text{s}}}^{*} {|}{\mathbf{g}}_{{\text{s}}} } \right)\; \propto \exp \left( { - \frac{1}{{2\sigma_{{\text{p}}}^{2} }}\left( {g_{{\text{s}}}^{*} - \mu_{{\text{p}}} } \right)^{2} } \right),$$

(58)

where

$$\begin{gathered} \mu_{{\text{p}}} = \frac{{\sigma_{{{\text{MAP}}}}^{2} {{\varvec{\Phi}}}^{{\text{T}}} ({\mathbf{x}}^{*} ){{\varvec{\Omega}}}^{ - 1} {{\varvec{\Sigma}}}^{ - 1} {{\varvec{\upmu}}}}}{{\sigma_{{{\text{MAP}}}}^{2} - {{\varvec{\Phi}}}({\mathbf{x}}^{*} )^{{\text{T}}} {{\varvec{\Omega}}}^{ - 1} {{\varvec{\Phi}}}({\mathbf{x}}^{*} )}} \hfill \\ \sigma_{{\text{p}}}^{2} = \left( {\frac{1}{{\sigma_{{{\text{MAP}}}}^{2} }} - \frac{{{{\varvec{\Phi}}}({\mathbf{x}}^{*} )^{{\text{T}}} {{\varvec{\Omega}}}^{ - 1} {{\varvec{\Phi}}}({\mathbf{x}}^{*} )}}{{(\sigma_{{{\text{MAP}}}}^{2} )^{2} }}} \right)^{ - 1} . \hfill \\ \end{gathered}$$

(59)

According to the Sherman–Morrison formula, the following relationship can be obtained:

$${{\varvec{\Omega}}}^{ - 1} = {{\varvec{\Sigma}}} - \frac{{{\mathbf{\Sigma \Phi }}\left( {{\mathbf{x}}^{*} } \right){{\varvec{\Phi}}}^{{\text{T}}} \left( {{\mathbf{x}}^{*} } \right){{\varvec{\Sigma}}}}}{{\sigma_{{{\text{MAP}}}}^{2} + {{\varvec{\Phi}}}^{{\text{T}}} \left( {{\mathbf{x}}^{*} } \right){\mathbf{\Sigma \Phi }}\left( {{\mathbf{x}}^{*} } \right)}}.$$

(60)

Substituting Eq. (60) into Eq. (59), the final mean value and variance of the posterior predictive distribution can be derived as

$$\begin{gathered} \mu_{{\text{p}}} = {{\varvec{\upmu}}}^{{\text{T}}} {{\varvec{\Phi}}}({\mathbf{x}}^{*} ) \hfill \\ \sigma_{{\text{p}}}^{2} = \sigma_{{{\text{MAP}}}}^{2} + {{\varvec{\Phi}}}({\mathbf{x}}^{*} )^{{\text{T}}} {\mathbf{\Sigma \Phi }}({\mathbf{x}}^{*} ). \hfill \\ \end{gathered}$$

(61)

Appendix 4

The real function value of a new test sample x^* is noted as g_s(x^*), and its estimation is $g_{{\text{s}}}^{{\text{e}}} ({\mathbf{x}}^{*} )$. After the final posterior predictive distribution $p(g_{{\text{s}}}^{*} |{\mathbf{g}}_{{\text{s}}} )$ is obtained, it can be seen as the distribution of g_s(x^*), p(g_s|g_s). Thus, a loss function that represents the mean error between the real function value and the estimation can be defined as

$$L_{g} = \int {\left( {g_{{\text{s}}} \left( {{\mathbf{x}}^{*} } \right) - g_{{\text{s}}}^{{\text{e}}} \left( {{\mathbf{x}}^{*} } \right)} \right)^{2} p\left( {g_{\text{s}} |{\mathbf{g}}_{\text{s}} } \right)} {\text{d}}g_{\text{s}} .$$

(62)

The $g_{{\text{s}}}^{{\text{e}}} ({\mathbf{x}}^{*} )$ minimizing L_g is expected. To this end, the derivative of L_g is calculated to obtain the stationary point; thus, the following equation yields

$$\frac{{{\text{d}}L_{g} }}{{{\text{d}}g_{{\text{s}}}^{{\text{e}}} }} = 2\int {\left( {g_{{\text{s}}}^{{\text{e}}} \left( {{\mathbf{x}}^{*} } \right) - g_{{\text{s}}} \left( {{\mathbf{x}}^{*} } \right)} \right)p\left( {g_{\text{s}} |{\mathbf{g}}_{\text{s}} } \right)} {\text{d}}g_{\text{s}} = 0.$$

(63)

From Eq. (63), the expected $g_{{\text{s}}}^{{\text{e}}} ({\mathbf{x}}^{*} )$ can be obtained as

$$g_{{\text{s}}}^{{\text{e}}} \left( {{\mathbf{x}}^{*} } \right) = \int {g_{{\text{s}}} \left( {{\mathbf{x}}^{*} } \right)p\left( {g_{\text{s}} |{\mathbf{g}}_{\text{s}} } \right)} {\text{d}}g_{\text{s}} .$$

(64)

Eq. (64) is nothing but the mean value of the posterior predictive distribution.