Objective-sensitive principal component analysis for high-dimensional inverse problems

Abstract

We introduce a novel approach of data-driven dimensionality reduction for solving high-dimensional optimization problems, including history matching. Objective-Sensitive parameterization of the argument accounts for the corresponding change of objective function value. The result is achieved via an extension of the conventional loss function, which only quantifies approximation error over realizations. This paper contains three instances of such an approach based on Principal Component Analysis (PCA). Gradient-Sensitive PCA (GS-PCA) exploits a linear approximation of the objective function. Two other approaches solve the problem approximately within the framework of stationary perturbation theory (SPT). All the algorithms are verified and tested with a synthetic reservoir model. The results demonstrate improvements in parameterization quality regarding the reveal of the unconstrained objective function minimum. Also, we provide possible extensions and analyze the overall applicability of the Objective-Sensitive approach, which can be combined with modern parameterization techniques beyond PCA.

Appendices

Appendix: aGS-PCA evaluation

Perturbed eigenproblem evaluation

To obtain an approximate solution for GS-PCA, we consider the objective-sensitive optimal decomposition problem with a sensitivity parameter ε, which satisfies a small perturbation criterion.

$$ \begin{array}{@{}rcl@{}} && \{ \boldsymbol{\varphi}_{i} \}_{i=1}^{N} = \arg \min_{\boldsymbol{\varphi}_{~}} <||\boldsymbol{\mu}_{N \text{r}}||_{W}^{2}> \end{array} $$

(30)

$$ \begin{array}{@{}rcl@{}} &&\text{s.t.}~ \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{W} = \delta_{ij} ,~ \varepsilon \geqslant 0 \end{array} $$

$$ \begin{array}{@{}rcl@{}} W = I + \varepsilon \boldsymbol{g}_{\boldsymbol{\eta}}^{T} \boldsymbol{g}_{\boldsymbol{\eta}} \end{array} $$

(31)

$$ \begin{array}{@{}rcl@{}} 0 \leqslant \varepsilon ||\boldsymbol{g}_{\boldsymbol{\eta}}||_{}^{2} \ll 1 \end{array} $$

(32)

Next, we express a residual norm $||\boldsymbol {\mu }_{N \text {r}}||_{W}^{2}$ in terms of principal components φ_i and decomposition coefficients a_i.

$$ ||\boldsymbol{\mu}_{N \text{r}}||_{W}^{2} = \sum\limits_{i>N} \sum\limits_{j>N} a_{i} a_{j} \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{W} $$

(33)

Assuming that the required solution is the first-order correction of standard principal components, we neglect higher-order terms of the scalar product $\left <\boldsymbol {\varphi }_{i},\boldsymbol {\varphi }_{j} \right >_{W}$.

$$ \begin{array}{@{}rcl@{}} \boldsymbol{\varphi}_{k} = \boldsymbol{\varphi}_{k}^{0}+\boldsymbol{\varphi}_{k}^{1} \end{array} $$

(34)

$$ \begin{array}{@{}rcl@{}} \left<\boldsymbol{\varphi}_{i}^{0},\boldsymbol{g}_{\boldsymbol{\eta}} \right> = b_{i} \end{array} $$

(35)

$$ \begin{array}{@{}rcl@{}} \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{W} \approx \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right> + \varepsilon b_{i} b_{j} \end{array} $$

(36)

After that, we express the mean product < a_ia_j > in terms of principal components φ_i, second moment K, scalar product matrix W, and derive a corresponding expression for the mean residual norm $<\|\boldsymbol {\mu }_{{N_{r}}}\|^{2}_{W}>$.

$$ \begin{array}{@{}rcl@{}} <a_{i} a_{j}> = \boldsymbol{\varphi}_{i}^{T} W <\boldsymbol{\mu} \boldsymbol{\mu}^{T}> W \boldsymbol{\varphi}_{j} = \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{W {K} W} \end{array} $$

(37)

$$ \begin{array}{@{}rcl@{}} <||\boldsymbol{\mu}_{N \text{r}}||_{W}^{2}> \approx \sum\limits_{i>N} \sum\limits_{j>N} (\left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right> + \varepsilon b_{i} b_{j}) \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{W {K}W} \end{array} $$

(38)

To achieve a sufficient simplicity of the approximate decomposition problem, we additionally neglect first-order perturbations in the obtained scalar product matrix WKW and the problem constraint.

$$ \begin{array}{@{}rcl@{}} W {K}W = {K} + \varepsilon (G {K} + {K}G) + \varepsilon^{2} G {K}G \approx {K} \end{array} $$

(39)

$$ \begin{array}{@{}rcl@{}} \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{I+\varepsilon G} = \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right> + \varepsilon \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{G} \approx \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right> = \delta_{ij} \end{array} $$

(40)

$$ \begin{array}{@{}rcl@{}} && \{ \boldsymbol{\varphi}_{i} \}_{i=1}^{N} = \arg \min_{\boldsymbol{\varphi}_{}} \sum\limits_{i>N} \sum\limits_{j>N} (\left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right> + \varepsilon b_{i} b_{j}) \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{{K}}\\ &&\text{s.t.}~ \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right> = \delta_{ij} \end{array} $$

(41)

Such constraint optimization implies the minimization of a corresponding Lagrangian.

$$ \begin{array}{@{}rcl@{}} &&F_{i} = \sum\limits_{j>N}(\delta_{ij} + \varepsilon b_{i} b_{j})\left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{{K}}\\ &&L = \sum\limits_{i>N}\left[ F_{i} - \sigma_{i}(\left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right> - 1) \right] \rightarrow \min \end{array} $$

(42)

Evaluating the necessary condition of constraint extremum,

$$ \begin{array}{@{}rcl@{}} \frac{\partial}{\partial \boldsymbol{\varphi}_{k}} \left<\boldsymbol{\varphi}_{i},\boldsymbol{\varphi}_{j} \right>_{{{{\varTheta}}}} = 2 \delta_{jk} {{{\varTheta}}} \boldsymbol{\varphi}_{i} \end{array} $$

(43)

$$ \begin{array}{@{}rcl@{}} \frac{\partial L}{\partial \boldsymbol{\varphi}_{k}} = 2 ({K}\boldsymbol{\varphi}_{k} + \varepsilon b_{k} \sum\limits_{i} b_{i} {K} \boldsymbol{\varphi}_{i} - \sigma_{k} \boldsymbol{\varphi}_{k}) = 0 \end{array} $$

(44)

we derive a perturbed eigenproblem associated with the approximate GS-PCA.

$$ {K}\boldsymbol{\varphi}_{k} + \varepsilon b_{k} \sum\limits_{i} b_{i} {K} \boldsymbol{\varphi}_{i} = \sigma_{k} \boldsymbol{\varphi}_{k} $$

(45)

Perturbed eigenproblem solution

A first-order term $\boldsymbol {\varphi }_{i}^{1}$ can be represented as a linear combination of unperturbed principal components given by a transform coefficients α_ij.

$$ \begin{aligned} &\boldsymbol{\varphi}_{k}^{1} = \sum\limits_{j} \alpha_{kj} \boldsymbol{\varphi}_{j}^{0} ,~ \alpha_{kk} = 0\\ &\sigma_{k} = {\sigma_{k}^{0}} + {\sigma_{k}^{1}} \end{aligned} $$

(46)

Given the known properties of unperturbed eigenvectors, we substitute decomposed vectors φ in the perturbed eigenproblem and project the expression onto an unperturbed vector $\boldsymbol {\varphi }_{n}^{0}$.

$$ \begin{array}{@{}rcl@{}} {K}\boldsymbol{\varphi}_{k}^{0} = {\sigma_{k}^{0}} \boldsymbol{\varphi}_{k}^{0} \end{array} $$

(47)

$$ \begin{array}{@{}rcl@{}} \sum\limits_{j} \alpha_{kj} {\sigma_{j}^{0}} \boldsymbol{\varphi}_{j}^{0} + \varepsilon b_{k} \sum\limits_{i} b_{i} {\sigma_{i}^{0}} \boldsymbol{\varphi}_{i}^{0} = {\sigma_{k}^{0}} \sum\limits_{j} \alpha_{kj} \boldsymbol{\varphi}_{j}^{0} + {\sigma_{k}^{1}} \boldsymbol{\varphi}_{k}^{0} \end{array} $$

(48)

$$ \begin{array}{@{}rcl@{}} \alpha_{kn} {\sigma_{n}^{0}} + \varepsilon b_{k} b_{n} {\sigma_{n}^{0}} = {\sigma_{k}^{0}} \alpha_{kn} + {\sigma_{k}^{1}} \delta_{kn} \end{array} $$

(49)

Finally, the required expressions for the unknowns ${\sigma _{n}^{1}}$ and α_kn are obtained from two cases of a relation between indices k and n.

$$ \begin{array}{@{}rcl@{}} k = n : \varepsilon b_{n} b_{n} {\sigma_{n}^{0}} = {\sigma_{n}^{1}} \end{array} $$

(50)

$$ \begin{array}{@{}rcl@{}} k \neq n : \alpha_{kn} {\sigma_{n}^{0}} + \varepsilon b_{k} b_{n} {\sigma_{n}^{0}} = {\sigma_{k}^{0}} \alpha_{kn} \end{array} $$

(51)

In this work, we additionally perform Gram – Schmidt orthogonalization on resulting vectors in terms of a scalar product $\left <\boldsymbol {\varphi }_{i},\boldsymbol {\varphi }_{j} \right >_{W}$.