Analytic regularity and stochastic collocation of high-dimensional Newton iterates

Abstract

In this paper, we introduce concepts from uncertainty quantification (UQ) and numerical analysis for the efficient evaluation of stochastic high-dimensional Newton iterates. In particular, we develop complex analytic regularity theory of the solution with respect to the random variables. This justifies the application of sparse grids for the computation of statistical measures. Convergence rates are derived and are shown to be subexponential or algebraic with respect to the number of realizations of random perturbations. Due to the accuracy of the method, sparse grids are well suited for computing low-probability events with high confidence. We apply our method to the power flow problem. Numerical experiments on the non-trivial, 39-bus New England power system model with large stochastic loads are consistent with the theoretical convergence rates. Moreover, compared with the Monte Carlo method, our approach is at least 10¹¹ times faster for the same accuracy.

Appendix: A Analyticity regions for random generators and loads

We examine the case of random generators and loads to obtain convergence rates of the sparse grid. The task is to synthesize an analyticity region Ψ for the extended Newton iteration to converge. We only check that the conditions of Theorem 7 are satisfied and assume that 6 is satisfied. A full analysis will be done in a future work. Thus, we synthesize the region of analyticity for Ψ by checking that:

$$ \| \mathbf{G}(\boldsymbol{\alpha}^{0},\mathbf{g}) \| < \frac{\delta_{e}}{\varkappa_{e}} - \frac{\delta}{\varkappa}, $$

whenever g ∈Ψ. Without loss of generality, assume that the first τ ≤ m buses contain stochastic power generators (or loads) with active power $P_{1}({\omega }):= (q_{1} + v_{1,R} + iv_{1,I}) c_{1} + a_{1}, ,\dots ,P_{\tau }({\omega }):= (q_{\tau } + v_{q,R} + iv_{q,I})c_{\tau } + a_{\tau }$ and reactive power $Q_{1}(\omega ):=(q_{{\tau }+1} + v_{\tau +1,R} + iv_{{\tau }+1,I}) c_{{\tau }+1} + a_{\tau +1},\dots ,$Q_τ(ω) := (q_N + v_N,R + iv_N,I)c_N + a_N. The random vector q ∈Γ is assumed to be stochastic with joint distribution ρ(q) and for all $k=1,\dots , N$v_k := v_k,R + iv_k,I is the complex extension of each q_k ∈Γ_k. Furthermore, for $k = 1, \dots , N$, the variables $a_{k} \in \mathbb {R}$ and $c_{k} \in \mathbb {R}^{+}$, where a_k + c_k and a_k indicate the maximum and minimum range of the stochastic perturbation. Thus, we have:

$$ \mathbf{f}(\boldsymbol{\alpha}_{0}) := \left[ \begin{array}{c} P_{1}(\mathbf{x}_{0}) - P_{1}({\omega})\\ {\vdots}\\ P_{\tau}(\mathbf{x}_{0}) - P_{\tau}({\omega})\\ P_{\tau+1}(\mathbf{x}_{0}) - P_{\tau+1}\\ {\vdots}\\ \displaystyle\frac{P_{m}(\mathbf{x}_{0}) - P_{m}} {Q_{1}(\mathbf{x}_{0}) - Q_{1}({\omega})}\\ {\vdots}\\ Q_{\tau}(\mathbf{x}_{0}) - Q_{\tau}({\omega})\\ Q_{\tau+1}(\mathbf{x}_{0}) - Q_{\tau+1}\\ {\vdots}\\ Q_{m}(\mathbf{x}_{0}) - Q_{m} \end{array} \right], $$

$$ {\text{Re}}{\mathbf{f}(\boldsymbol{\alpha}_{0})} = \left[ \begin{array}{c} P_{1}(\mathbf{x}_{0}) - (q_{1} + v_{1,R})c_{1} + a_{1}\\ {\vdots}\\ P_{\tau}(\mathbf{x}_{0}) - (q_{\tau} + v_{q,R})c_{\tau} + a_{\tau}\\ P_{\tau+1}(\mathbf{x}_{0}) - P_{\tau+1}\\ {\vdots}\\ P_{m}(\mathbf{x}_{0}) - P_{m}\\ Q_{1}(\mathbf{x}_{0}) - (q_{{\tau}+1} + v_{q+1,R})c_{{\tau}+1} + a_{{\tau}+1}\\ {\vdots}\\ Q_{{\tau}}(\mathbf{x}_{0}) - (q_{N} + v_{N,R})c_{N} + a_{N}\\ Q_{{\tau}+1}(\mathbf{x}_{0}) - Q_{\tau+1}\\ {\vdots}\\ Q_{m}(\mathbf{x}_{0}) - Q_{m} \end{array} \right], $$

$$ {\text{Im}}{\mathbf{f}(\boldsymbol{\alpha}_{0})} = -\left[ \begin{array}{ccccccccccccc} v_{1,I}c_{1} & {\dots} & v_{{\tau},I}c_{\tau} & 0 & {\dots} & 0 & | & v_{{\tau}+1,I}c_{{\tau}+1} & {\dots} & v_{N,I}c_{N} & 0 & {\dots} & 0 \end{array} \right]^{T}, $$

and therefore

$$ \mathbf{P}_{P} =- \left[ \begin{array}{c} (v_{1,R} + v_{1,I})c_{1} \\ {\vdots} \\ (v_{{\tau},R} + v_{{\tau},I})c_{{\tau}} \\ 0 \\ \vdots\\ 0 \end{array} \right], \mathbf{P}_{Q} =- \left[ \begin{array}{c} (v_{{\tau}+1,R} + v_{{\tau}+1,I})c_{{\tau}+1} \\ {\vdots} \\ (v_{N,R} + v_{N,I})c_{N} \\ 0 \\ \vdots\\ 0 \end{array} \right], $$

$\mathbf {P} = \begin {bmatrix} \mathbf {P}_{P} \\ \mathbf {Q}_{Q} \end {bmatrix}$ and thus $\| \mathbf {G} \|^{2}_{2} = {\sum }_{k=1}^{N} (v_{k,R} + v_{k,I})^{2}{c^{2}_{k}} + {\sum }_{k=1}^{N} v^{2}_{k,I}{c^{2}_{k}}.$ The last equality is due to the integral remainder form of Taylor’s theorem. Let v_k,R = v_R/c_k and v_k,I = v_I/c_k for $k = 1,\dots , 2N$, where $v_{R}, v_{I} \in \mathbb {R}$, then for any 𝜖 > 0

$$ \begin{array}{ll} \| \mathbf{G} \|^{2}_{2} & = N{v_{R}^{2}} + 2N{v_{I}^{2}} + 2Nv_{R}v_{I}\\ &\leq {v_{R}^{2}} N(1 + 2 \epsilon) + {v_{I}^{2}}N(2 + \epsilon^{-1}/2) \\ &\leq \left( \frac{\delta_{e}}{\varkappa_{e}} - \frac{\delta}{\varkappa} \right)^{2}. \end{array} $$

The last inequality is obtained by using Cauchy’s inequality. Let $\gamma _{e} := \frac {\delta _{e}}{\varkappa _{e}} - \frac {\delta }{\varkappa }$;

thus, the inequality:

$$ \frac{{v_{R}^{2}}}{\alpha^{2}} + \frac{{v_{I}^{2}}}{{\upbeta}^{2}} \leq 1, $$

(1)

where $\alpha ^{2} := \frac {{\gamma _{e}^{2}}}{N(1+2\epsilon )}$ and ${\upbeta }^{2} := \frac {{\gamma _{e}^{2}}}{N(2 + \epsilon ^{-1}/2)}$, forms an elliptical region ${\Sigma } \subset \mathbb {C}$ (see Fig. 7) such that ∥G∥₂ ≤ γ_e.

Fig. 7

Embedding of Bernstein ellipse ${\mathcal E}_{\sigma }$ in the domain Φ

Consider the region Φ := {g = q + v_R + iv_I|q ∈ [− 1, 1], (v_R,v_I) ∈Σ} where ∥G∥₂ ≤ γ_e is satisfied. Suppose that we want to embed a Bernstein ellipse ${\mathcal E}_{\sigma }$ in Φ. To achieve this, consider the foci points of ${\mathcal E}_{\sigma }$ at − 1 and 1. It is not hard to show that $\frac {e^{\sigma } - e^{-\sigma }}{2} \geq \frac {e^{\sigma } + e^{-\sigma }}{2} - 1$ for σ > 0. At the foci point ± 1, trace the ellipse from Eq. 1 and set $\upbeta = \frac {e^{\sigma } - e^{-\sigma }}{2}$ (see Fig. 7).

Choose an 𝜖 > 0 such that ${\mathcal E}_{\sigma }$ is embedded in Φ by solving the following equation:

$$ \frac{\alpha}{\upbeta} = \sqrt{\frac{ 4 + \epsilon^{-1}}{2(1 + 2 \epsilon)}} $$

leading to

$$ \epsilon = \frac{(c^{2} + 4)^{\frac{1}{2}} - c + 2}{4c} > 0, $$

where c := (α/β)² > 0. Pick 𝜖 > 0 such that c = (α/β)² = 1. Pick σ > 0 such that $\frac {e^{\sigma } - e^{-\sigma }}{2} = \upbeta$. This leads to

$$ \sigma = \log(\upbeta + ({\upbeta}^{2} + 1)^{\frac{1}{2}}) $$

and $\frac {e^{\sigma } + e^{-\sigma }}{2} - 1 \leq \upbeta = \alpha$. The region bounded by the ellipse ${\mathcal E}_{\sigma }$ is therefore embedded in Φ.

A polyellipse in C^N can now be constructed such that ∥G∥₂ ≤ γ_e. Recall that v_k,R = v_R/c_k, and v_k,I = v_I/c_k for $k = 1,\dots , N$ and consider the regions Ψ_k := {g = q + v_k,R + iv_k,I|q ∈ [− 1, 1], (v_R,v_I) ∈Σ}. By following the procedure for embedding ${\mathcal E}_{\sigma }$ in Φ for each $k = 1,\dots ,N$, an ellipse ${\mathcal E}_{\varrho _{k}}$ can be embedded in Ψ_k with:

$$ \varrho_{k} := \log\left( \frac{\upbeta}{c_{k}} + \left( \frac{{\upbeta}^{2}}{{c_{k}^{2}}} + 1 \right)^{\frac{1}{2}}\right). $$

The polyellipse ${\mathcal E}_{\varrho _{1},\dots ,\varrho _{N}} := {\mathcal E}_{\varrho _{1}} \times {\dots } \times {\mathcal E}_{\varrho _{N}}$ is embedded in ${\Psi }_{1} \times {\dots } \times {\Psi }_{N}$. Thus for any $\text {g} \in {\mathcal E}_{\varrho _{1},\dots ,\varrho _{N}}$

$$ \| \mathbf{G}(\boldsymbol{\alpha}_{0},\mathbf{g}) \| \leq \gamma_{e}. $$

With the region bounded by the polyellipse ${\mathcal E}_{\varrho _{1},\dots ,\varrho _{N}}$ the convergence rate of the sparse grid can be estimated with respect to the magnitude of the coefficients c_k, for $k = 1, \dots , N$.

Remark 1

From this analysis we observe that the size of the analyticity region of Ψ depends directly on

$$ \upbeta = \sqrt{\frac{{\gamma_{e}^{2}}}{N(2 + \epsilon^{-1}/2)}}, $$

where $\epsilon = \frac {\sqrt {5} + 1}{4}$. The size of the region Ψ decays as square root with respect to the number of stochastic dimensions N, thus reducing the convergence rate of the sparse grid.

Example 1

Consider the 3-bus simple power system with stochastic load and generator based on Example 10.6 in [8] and Fig. ??. Bus 1 is the slack bus with 1 0^∘. Bus 2 voltage is fixed as V₂ = 1.05 p.u. and contains a stochastic generator P_E = q₁c₁ + a₁, where q₁ ∈ [− 1, 1] with a₁ = 0.6661 p.u. and $c_{1} \in \mathbb {R}^{+}$; i.e., the generator is random within the range 0.6661 ± c₁. Bus 3 contains the random load with P_L + iQ_L = (q₂c₂ + a₂) + i(q₃c₃ + a₃), where q₂,q₃ ∈ [− 1, 1] with a₂ = 2.8653 p.u., a₃ = 1.2244 p.u. and $c_{2},c_{3} \in \mathbb {R}^{+}$, i.e., the load is random within the range 2.8653 ± c₂ of the active power and 1.2244 ± c₃ for the reactive power. The admittance matrix to the network is:

$$ \mathbf{Y}_{bus} := i \left( \begin{array}{ccc} -20 & 10 & 10 \\ 10 & -20 & 10 \\ 10 & 10 & -20 \end{array}\right). $$

The vector of unknowns is x := [𝜃₂,𝜃₃,V₃]^T and f(x,q) = [P₁(x) − q₁c₁ − a₁,P₂(x) − q₂c₂ − a₂,P₃(x) − q₃c₃ − a₃]^T for all q ∈Γ and therefore f(α,g) = [P₁(α)−(q₁+v_1,R+iv_1,I)c₁+a₁,P₂(α)−(q₂+v_2,R+iv_2,I)c₂+a₃,P₃(α)−(q₃+v_3,R+iv_3,I)c₃+a₃]^T for all g ∈Ψ. With q = 0 the Newton algorithm converges in 20 iterations to x^∗ = [− 5.2361 × 10^− 2,− 1.7445 × 10^− 1, 0.9500]^T with 10^− 15 tolerance. Furthermore,

$$ \mathbf{J}(\mathbf{x}^{*}) = \left( \begin{array}{ccc} 0.06542 &~ 0.03384 &~ 0.00135\\ 0.03384 &~ 0.07112 &~ 0.01127\\ 0.00128 &~ 0.01069 &~ 0.06549\\ \end{array}\right) \text{and} \varkappa = \| \mathbf{J}^{-1}(\mathbf{x}^{*}) \|_{2} = 0.1043. $$

Since there is no direct stochastic components (q₁,q₂,q₃) in the Jacobian matrix, then

$$ \begin{array}{@{}rcl@{}} \begin{array}{llll} \mathbf{J}(\mathbf{x}) &= \left[ \begin{array}{ccccc} 10.5 (cos(\theta_{2}) + V_{3} cos(\theta_{2} - \theta_{3})) & -10.5 V_{3} cos(\theta_{2} - \theta_{3}) \\ -10.5 V_{3} cos(\theta_{3} - \theta_{2}) & 10 V_{3} cos(\theta_{3}) + 10.5 V_{3} cos(\theta_{3} - \theta_{2}) \\ -10.5 V_{3} sin(\theta_{3} - \theta_{2}) & 10.5 V_{3} (sin(\theta_{3}) + sin(\theta_{3} - \theta_{2})) \end{array} \right. \\ & \left. \begin{array}{cccccc} 10.5 sin(\theta_{2} - \theta_{3}) \\ 10.5 sin(\theta_{3}) + 10.5 sin(\theta_{3} - \theta_{2}) \\ -(10 cos(\theta_{3}) +10.5 cos(\theta_{3} - \theta_{2}) - 39.96{V_{3}^{2}}) \end{array}\right]. \end{array} \end{array} $$

From the mean value theorem, we have that:

$$ \lambda = \sqrt{ {\sum}_{k,l=1}^{m} \| \nabla \mathbf{J}_{k,l}(\mathbf{x}) \|_{L^{\infty}(D \times {\Gamma})} } < \infty, $$

where D is a bounded set, J_k,l(x) is the k^th row and l^th column entry of the Jacobian matrix J(x).

With the initial condition $\mathbf {x}_{0} = \mathbf {x}^{*}(\mathbf {0})$, for small enough coefficients c₁,c₂, and c₃ we have that:

• (i) $\| \mathbf {J}^{-1}(\mathbf {x}_{0}) \| \leq \varkappa = 0.1043$.

• (ii) Furthermore, since f is continuous and f(x₀,0) = 0, then $\| \mathbf {J}^{-1}(\mathbf {x}_{0}) \mathbf {f}(\mathbf {x}_{0},\mathbf {q})\| \leq \delta < \infty$, where δ is arbitrarily small.

• (iii) It follows that h < 1 for all $\mathbf {x} \in \overline {B(\mathbf {x}^{*}(0),t^{*})} \subset D$ where $t^{*} = \frac {2}{h}(1 - \sqrt {1 - h})\delta$.

• v From the Newton–Kantorovich theorem, the iteration converges for all q ∈Γ and

  $$ \| V_{3}(\mathbf{q}) \|_{L^{\infty}({\Gamma})} = \sup_{\mathbf{x} \in \overline{B(\mathbf{x}_{0},t^{*})}} |\mathbf{x}[3]| \leq t^{*} + \|\mathbf{x}^{*}(\mathbf{0})\|_{l^{2}(\mathbb{R}^{m})}. $$

Now, pick δ_e > 0 such that δ < δ_e and also pick ϰ_e = ϰ, λ_e = λ such that h < h_e ≤ 1. From the random load analysis, we have $\gamma _{e} := \frac {\delta _{e}}{\varkappa _{e}} - \frac {\delta }{\varkappa }$ and

$$ \| \mathbf{G}(\boldsymbol{\alpha}_{0},\mathbf{g}) \| \leq \gamma_{e} $$

for all $\mathbf {g} \in {\mathcal E}_{\varrho _{1},\varrho _{2},\varrho _{3}}$ where

$$ \upbeta = \left( \frac{(1 + \sqrt{5})}{6(2 + \sqrt{5}) } \right)^{\frac{1}{2}} \gamma_{e} $$

and for $k = 1,\dots ,3$

$$ \varrho_{k} := \log\left( -\frac{\upbeta}{c_{k}} + \left( \frac{{\upbeta}^{2}}{{c_{k}^{2}}} + 1 \right)^{\frac{1}{2}}\right). $$

Assuming that Theorem 6 is satisfied, then from Theorem 7 it follows that whenever g ∈Ψ then

$$ \| \mathbf{J}(\mathbf{x}_{0})^{-1} \mathbf{f}(\boldsymbol{\alpha}_{0},\mathbf{g}) \| \leq \delta_{e}, $$

the limit of the Newton iteration converges and is holomorphic in ${\mathcal E}_{\varrho _{1},\varrho _{2},\varrho _{3}} \subset {\Psi } \subset \mathbb {C}^{3}$.