Performance enhancement of Gauss-Newton trust-region solver for distributed Gauss-Newton optimization method

Abstract

Distributed Gauss-Newton (DGN) optimization method has been proved very efficient and robust for history matching and uncertainty quantification (HM&UQ). The major bottleneck for performance enhancement is the expensive computational cost of solving hundreds of Gauss-Newton trust-region (GNTR) subproblems in each iteration. The original GNTR solver applies the less efficient iterative Newton-Raphson (NR) method using a derivative which requires solving a large-scale linear system twice in each NR iteration. Instead of using a less accurate linear proxy as in the iterative NR method, the nonlinear GNTR equation is first approximated with an inverse-quadratic (IQ) or a cubic-spline (CS) model, by fitting points generated in previous iterations without using any derivative. Then, the analytical (or numerical) solution of the IQ (or CS) model is used as the new proposal for the next iteration. The performances of the two new GNTR solvers are benchmarked against different methods on different sets of test problems with different numbers of uncertain parameters (ranging from 500 to 100,000) and different numbers of observed data (ranging from 100 to 100,000). In terms of efficiency and robustness, the two new GNTR solvers have comparable performance, and they outperform other methods we tested, including the well-known direct and iterative trust-region solvers of the GALAHAD optimization library. Finally, the proposed GNTR solvers have been implemented in our in-house distributed HM&UQ system. Our numerical experiments indicate that the DGN optimizer using the newly proposed GNTR solver performs quite stable and is effective when applied to real-field history matching problems.

Appendix: Cubic spline interpolation without derivative

Let $c_{i}\left (\lambda \right )$ be a cubical function that interpolates the two points, p_i− 1 = p(λ_i− 1) and p_i = p(λ_i) with λ_i− 1 < λ_i, i.e.

$$ \begin{array}{@{}rcl@{}} c_{i}\left( \lambda \right)&=&\left( 1-t \right)p_{i-1}+tp_{i}+t\left( 1-t \right)[a_{i}\left( 1-t \right)+b_{i}t],\\ &&\text{for}~i=1,2,{\ldots} ,m. \end{array} $$

(33)

In Eq. 33, t = (λ − λ_i− 1)δ_i− 1, $a_{i}=\frac {k_{i-1}}{\delta _{i-1}}-{\Delta } p_{i-1}$, and $b_{i}=-\frac {k_{i}}{\delta _{i-1}}{+{\Delta } }p_{i-1}$. Here, we define δ_i− 1 = 1/(λ_i − λ_i− 1) and Δp_i− 1 = p_i − p_i− 1. $k_{i}=c_{i}^{\prime }(\lambda _{i})$ is the derivative of $c_{i}\left (\lambda \right )$ evaluated at λ_i. The first- and the second-order derivatives of $c_{i}\left (\lambda \right )$ are, respectively

$$ \begin{array}{@{}rcl@{}} c_{i}^{\prime}\left( \lambda \right)&=&\left\{ {\Delta} p_{i-1}{+(1-2}t)[a_{i}\left( 1-t \right){+}b_{i}t]\right.\\ &&\left.+(b_{i}-a_{i})t\left( 1-t \right) \right\}\delta_{i-1}. \end{array} $$

(34)

$$ c_{i}^{\prime\prime}\left( \lambda \right)=2\left\{ b_{i}-2a_{i}+3(a_{i}-b_{i})t \right\}\delta_{i-1}^{\mathrm{2}}. $$

(35)

Obviously, the first-order derivative is continuous at point λ_i; i.e., $k_{i}=c_{i}^{\prime }\left (\lambda _{i} \right )=c_{i{+1}}^{\prime }(\lambda _{i})$ for i = 1, 2,…,m − 1. At the two ending points, we have $k_{\mathrm {0}}=c_{\mathrm {1}}^{\prime }\left (\lambda _{\mathrm {0}} \right )$ and $k_{m}=c_{m}^{\prime }\left (\lambda _{m} \right )$. The second-order derivative of $c_{i}^{\prime \prime }\left (\lambda \right )$ at the two points λ_i− 1 and λ_i are, respectively

$$ c_{i}^{\prime\prime}\left( \lambda_{i-1} \right)=6{\Delta} p_{i-1}\delta_{i-1}^{\mathrm{2}}-(2k_{i-1}{+}k_{i})\delta_{i-1}. $$

(36)

$$ c_{i}^{\prime\prime}\left( \lambda_{i} \right)=-6{\Delta} p_{i-1}\delta_{i-1}^{\mathrm{2}}+(k_{i-1}+2k_{i})\delta_{i-1}. $$

(37)

The continuous second-order derivative at point λ_i requires $c_{i}^{\prime \prime }\left (\lambda _{i} \right )=c_{i+1}^{\prime \prime }\left (\lambda _{i} \right )$, for i = 1, 2,…,m − 1, i.e.

$$ \delta_{i-1}k_{i-1}+2\left( \delta_{i-1}{+}\delta_{i} \right)k_{i}{+}\delta_{i}k_{i+1}=3({\Delta} p_{i-1}\delta_{i-1}^{\mathrm{2}}+{\Delta} p_{i}\delta _{i}^{\mathrm{2}}). $$

(38)

The natural spline conditions also require $c_{\mathrm {1}}^{\prime \prime }\left (\lambda _{\mathrm {0}} \right )=c_{m}^{\prime \prime }\left (\lambda _{m} \right )=0$, i.e.

$$ \mathrm{2}\delta_{\mathrm{0}}k_{\mathrm{0}}+\delta_{\mathrm{0}}k_{\mathrm{1}}=3{\Delta} p_{\mathrm{0}}\delta_{\mathrm{0}}^{\mathrm{2}}. $$

(39)

$$ \delta_{m-1}k_{m-1}+{\mathrm{2}\delta }_{m-1}k_{m}=3{\Delta} p_{m-1}\delta_{m-1}^{\mathrm{2}}. $$

(40)

Therefore, the m + 1 unknown parameters, k_i for i = 0, 1, 2,…,m, can be solved from the tridiagonal linear equation

$$ AK=C. $$

(41)

In Eq. 41, A = [a_i,j] is an (m + 1) × (m + 1) triangular matrix with a_0,0 = 2δ₀, a_m,m = 2δ_m− 1, a_i,i− 1 = a_i− 1,i = δ_i− 1 and a_i,i = 2(δ_i− 1 + δ_i) for i = 1, 2,…,m − 1. K = [k₀,k₁,…,k_m]^T, and C = [c_i] is an (m + 1)-dimensional vector with $c_{0} = 3{\Delta } p_{0} {\delta }_{0}^{2}$, $c_{m} = 3 {\Delta } p_{m-1} {\delta }_{m-1}^{2}$ and $c_{i} = 3({\Delta } p_{i-1} {\delta }_{i-1}^{2} + {\Delta } p_{i} {\delta }_{i}^{2})$ for i = 1, 2,…,m − 1.

In case of interpolating the three points of λ₀, λ₁, and λ₂ with λ₀ < λ₁ < λ₂, the solution of Eq. 41 gives

$$ k_{\mathrm{1}}=\frac{{{\Delta} }p_{\mathrm{0}}\delta_{\mathrm{0}}^{\mathrm{2}}+{\Delta} p_{\mathrm{1}}\delta_{\mathrm{1}}^{\mathrm{2}}}{\delta_{\mathrm{0}}{+}\delta _{\mathrm{1}}}. $$

(42)

$$ k_{\mathrm{0}}=1.5{\Delta} p_{\mathrm{0}}\delta_{\mathrm{0}}-0.5k_{\mathrm{1}}. $$

(43)

$$ k_{\mathrm{2}}=1.5{\Delta} p_{\mathrm{1}}\delta_{\mathrm{1}}-0.5k_{\mathrm{1}}. $$

(44)

Then, we can find the solution of the cubic spline function $c\left (\lambda \right )=0$ iteratively, e.g., using the Newton-Raphson method. Because both $c\left (\lambda \right )$ and $c^{\prime }\left (\lambda \right )$ can be evaluated analytically, the computational cost of finding the solution $c\left (\lambda \right )=0$ is negligible when compared to the cost of evaluating $\pi \left (\lambda \right )$. In addition to $c\left (\lambda \right )$ and $c^{\prime }\left (\lambda \right )$, we can also analytically evaluate $c^{\prime \prime }\left (\lambda \right )$, e.g., using Eq. 35.