# Asymptotic convergence of spectral inverse iterations for stochastic eigenvalue problems

- 277 Downloads

## Abstract

We consider and analyze applying a spectral inverse iteration algorithm and its subspace iteration variant for computing eigenpairs of an elliptic operator with random coefficients. With these iterative algorithms the solution is sought from a finite dimensional space formed as the tensor product of the approximation space for the underlying stochastic function space, and the approximation space for the underlying spatial function space. Sparse polynomial approximation is employed to obtain the first one, while classical finite elements are employed to obtain the latter. An error analysis is presented for the asymptotic convergence of the spectral inverse iteration to the smallest eigenvalue and the associated eigenvector of the problem. A series of detailed numerical experiments supports the conclusions of this analysis.

## Mathematics Subject Classification

65C20 65N12 65N15 65N25 65N30## 1 Introduction

During the recent years numerical solution of stochastic partial differential equations (sPDE) has attracted a lot of attention and become a well-established field. However, the field of stochastic eigenvalue problems (sEVP) and their numerical solution is still in its infancy. It is natural that, after the source problem, more effort is put on addressing the eigenvalue problem.

A few different algorithms have recently been suggested for computing approximate eigenpairs of sEVPs. As with sPDEs, the solution methods are typically divided into intrusive and non-intrusive ones. A benchmark for non-intrusive methods is the sparse collocation algorithm suggested and thoroughly analyzed by Andreev and Schwab [1]. An attempt towards a Galerkin-based (intrusive) method was made by Verhoosel et al. [20], though this method omits uniform normalization of the eigenmodes. Very recently Meidani and Ghanem proposed a spectral power iteration, in which the eigenmodes are normalized using a quadrature rule over the parameter space [16]. The algorithm has been further developed and studied by Sousedík and Elman [19]. However, neither of the papers present a comprehensive error analysis for the method.

Inspired by the original method of Meidani and Ghanem we have suggested a purely Galerkin-based spectral inverse iteration, in which normalization of the eigenmodes is achieved via solution of a simple nonlinear system [11]. This method, and its generalization to a spectral subspace iteration, is the focus of the current paper. Although the algorithms in [16, 19] differ from ours in the way normalization is performed, the basic principles are still the same and hence our results on convergence should apply to these methods as well.

In this work we consider computing eigenpairs of an elliptic operator with random coefficients. We assume a physical domain \(D \subset {\mathbb {R}}^d\) and, in order to capture the random dimension of the system, a parameter domain \(\varGamma \subset {\mathbb {R}}^{\infty }\) with associated measure \(\nu \). One may think of a parametrization that arises from Karhunen-Loève representations of the random coefficients in the system, for instance. Discretization in space is achieved by standard FEM and associated with a discretization parameter *h*, whereas discretization in the random dimension is achieved using collections of certain multivariate polynomials. These collections are represented by multi-index sets \(\mathcal {A}_{\epsilon }\) of increasing cardinality \(\# \mathcal {A}_{\epsilon }\) as \(\epsilon \rightarrow 0\).

The first term in the formulas (1) and (2) is justified by standard theory for Galerkin approximation of eigenvalue problems, a simple consequence of which we have recapped in Theorem 1. The second term can be deduced from Theorem 2, which bounds the Galerkin approximation errors by residuals of certain polynomial approximations of the solution. Using best *P*-term polynomial approximations, we see that these residuals are ultimately expected to decay at an algebraic rate \(r > 0\), see [5] and [7]. Finally, the third term follows from Theorem 3, which states that asymptotically the iterates of the spectral inverse iteration converge to a fixed point in geometric fashion. Here the analogy to classical inverse iteration is evident. Each of these three important steps that comprise the main result is separately verified through detailed numerical examples.

A variant of our algorithm for spectral subspace iteration is also presented. No analysis of this algorithm is given, but the numerical experiments support the conclusion that it converges towards the exact subspace of interest, and that the rate of convergence is analogous to what we would expect from classical theory. This is despite the fact that the individual eigenmodes, as defined by the pointwise order of magnitude of the eigenvalues, are not continuous functions over the parameter space due to an eigenvalue crossing. To the authors’ knowledge such a scenario has not yet been considered in the scientific literature.

The rest of the paper is organized as follows. Our model problem and its fundamental properties are assessed in Sects. 2 and 3. A detailed review of the discretization of the spatial and stochastic approximation spaces is given in Sect. 4. Analysis of the spectral inverse iteration, supported by thorough numerical experiments, is given in Sect. 5. Finally, the algorithm of spectral subspace iteration and numerical experiments of its convergence are presented in Sect. 6.

## 2 Problem statement

In this work we consider eigenvalue problems of elliptic operators with random coefficients. It is assumed that the random coefficients admit a parametrization with respect to countably many independent and bounded random variables. As a model problem we consider the eigenvalue problem of a diffusion operator with a random diffusion coefficient. It will be evident, however, that our methods and analysis in fact cover a much broader class of problems.

### 2.1 Model problem

*D*. The diffusion coefficient is assumed to be strictly uniformly positive and uniformly bounded, i.e., for some positive constants \(a_{\min }\) and \(a_{\max }\) it holds that

### 2.2 Parametrization of the random input

The usual convention is that the parametrization (5) results from a Karhunen-Loève expansion, which gives \(a(x, \omega )\) as a linear combination of the eigenfunctions of the associated covariance operator. The distinguishing feature of the Karhunen-Loève expansion compared to other linear expansions is that it minimizes the mean square truncation error [9].

### 2.3 Parametric eigenvalue problem and its variational formulation

*A*(

*y*) for \(y \in \varGamma \).

*b*(

*y*;

*u*,

*v*) is continuous and elliptic. Thus, as in [1, 11], we deduce that the problem (10) admits a countable number of real eigenvalues and corresponding eigenfunctions that form an orthogonal basis of \(L^2(D)\).

## 3 Analyticity of eigenmodes

A key issue in the analysis of parametric eigenvalue problems is that eigenvalues may cross within the parameter space. Here we first disregard this possibility and recap the main results from [1] for simple eigenvalues that are sufficiently well separated from the rest of the spectrum. In Sect. 6 we briefly comment on the case of possibly clustered eigenvalues and associated invariant subspaces.

- (i)
\(\mu (y)\) is simple as an eigenvalue of

*A*(*y*) for all \(y \in \varGamma \) and - (ii)
the minimum spectral gap \(\inf _{y \in \varGamma } {{\,\mathrm{dist}\,}}(\mu (y), \sigma (A(y)) \backslash \{\mu (y) \})\) is positive.

*y*.

### Proposition 1

*u*normalized so that \(|| u(y) ||_{L^2(D)} = 1\) for all \(y \in \varGamma \). For \(s \in {\mathbb {N}}\) assume that \(a_0 \in W^{s,\infty }(D)\) and the assumptions (6)–(8) hold for some \(p_0, p_s \in (0,1)\). Given \(\tau = (\tau _1, \tau _2, \ldots )\in \mathbb {R}_+^{\infty }\) define

*m*such that with \(C_2 > 0\) arbitrary and \(\tau \) given by

### Proof

This is analogous to Corollary 2 of Theorem 4 in [1]. \(\square \)

It is well known that for elliptic operators on a connected domain *D* the smallest eigenvalue is simple [12]. Thus, Proposition 1 may at least be applied for the smallest eigenvalue of problem (9).

## 4 Stochastic finite elements

Proposition 1, under sufficient assumptions, guarantees the existence of an analytic eigenpair for problem (9). It now makes sense to look for the eigenvalue in the space \(L^2_{\nu }(\varGamma )\) and the eigenfunction in the space \(L^2_{\nu }(\varGamma ) \otimes H^1_0(D)\). The space \(H^1_0(D)\) may be discretized by means of the traditional finite element method. For the discretization of \(L^2_{\nu }(\varGamma )\), we follow the usual convention in stochastic Galerkin methods and construct a basis of orthogonal polynomials of the input random variables. Orthogonal polynomials for various probability distributions exist and the use of these as the approximation basis has been observed to yield optimal rates of convergence [18, 21]. Here we consider uniformly distributed random variables which lead to the choice of tensorized Legendre polynomials.

### 4.1 Galerkin discretization in space

*l*.

### Theorem 1

### Proof

This follows from the theory of Galerkin approximation for variational eigenvalue problems. See Section 8 in [3] and Section 9 in [8]. \(\square \)

### 4.2 Legendre chaos

*p*. We will assume the normalization \({\mathbb {E}}[\varLambda _{\alpha }^2] = 1\) for all \(\alpha \in (\mathbb {N}_0^{\infty })_c\).

*v*in a series

### 4.3 Sparse polynomial approximation in the parameter domain

We proceed as in [5] and use best *P*-term approximations to prove convergence of the approximation error.

### Proposition 2

*H*be a Hilbert space. Assume that \(v \!: \varGamma \rightarrow H\) admits a complex-analytic extension in the region

### Proof

### 4.4 Stochastic Galerkin approximation of vectors and matrices

We now generalize the concept of sparse polynomial approximation to vector and matrix valued functions. Assume that the dimensions of the approximation spaces \(V_h\) and \(W_{\mathcal {A}}\) are *N* and *P* respectively. We denote by \(W_{\mathcal {A}}^N\) (or \(W_{\mathcal {A}}^{N \times N}\)) the space of functions \({\mathbf {v}}\! : \varGamma \rightarrow {\mathbb {R}}^N\) (or \(\mathbf {A} \! : \varGamma \rightarrow {\mathbb {R}}^{N \times N}\)) whose every component is in \(W_{\mathcal {A}}\). Whenever \({\mathbf {v}}\in W_{\mathcal {A}}^N\) and \(\alpha \in \mathcal {A}\) we set \(v_{\alpha i} = ({\mathbf {v}}_i)_{\alpha }\) and use \({\mathbf {v}}_{\alpha }\) to denote the vector of coefficients \(\{ v_{\alpha i} \}_{i \in J} \in {\mathbb {R}}^N\). Moreover, we associate any \(v \in W_{\mathcal {A}}\) with the array of coefficients \({\hat{v}} := \{ v_{\alpha } \}_{\alpha \in \mathcal {A}} \in {\mathbb {R}}^P\) and similarly any \({\mathbf {v}}\in W_{\mathcal {A}}^N\) with the array of coefficients \({\hat{{\mathbf {v}}}}:= \{ v_{\alpha i} \}_{\alpha \in \mathcal {A}, i \in J} \in {\mathbb {R}}^{PN}\).

### Remark 1

### Lemma 1

### Proof

## 5 Spectral inverse iteration

In this section we introduce the algorithm of spectral inverse iteration, analyze its asymptotic convergence, and present numerical examples to support our analysis. The spectral inverse iteration, see [11], can be considered as an extension of the classical inverse iteration to the case of parametric matrix eigenvalue problems. In the spectral version each of the elementary operations is computed in Galerkin sense via projecting to the sparse polynomial basis \(W_{\mathcal {A}}\). Optimal convergence of the algorithm requires that the eigenmode of interest, i.e., the smallest eigenvalue of the parametric matrix, is strictly nondegenerate.

### 5.1 Algorithm description

Fix a finite set of multi-indices \(\mathcal {A} \subset (\mathbb {N}_0^{\infty })_c\) and let \(P= \#\mathcal {A}\). The spectral inverse iteration for the system (16) is now defined in Algorithm 1. One should note that, if the projections in the algorithm were precise, the algorithm would correspond to performing classical inverse iteration pointwise over the parameter space \(\varGamma \). We expect the algorithm to converge to an approximation of the eigenvector corresponding to the smallest eigenvalue of the system.

### Algorithm 1

- (1)Solve \({\mathbf {v}}\in W_{\mathcal {A}}^N\) from the linear equation$$\begin{aligned} P_{\mathcal {A}} \left( \mathbf {K} {\mathbf {v}}\right) = \mathbf {M} {\mathbf {u}}^{(k-1)}. \end{aligned}$$(34)
- (2)Solve \(s \in W_{\mathcal {A}}\) from the nonlinear equation$$\begin{aligned} P_{\mathcal {A}} (s^2) = P_{\mathcal {A}} \left( || {\mathbf {v}}||_{{\mathbb {R}}^N_{\mathbf {M}}}^2 \right) . \end{aligned}$$(35)
- (3)Solve \({\mathbf {u}}^{(k)} \in W_{\mathcal {A}}^N\) from the linear equation$$\begin{aligned} P_{\mathcal {A}} \left( s {\mathbf {u}}^{(k)} \right) = {\mathbf {v}}. \end{aligned}$$(36)
- (4)
Stop if \(|| {\mathbf {u}}^{(k)} - {\mathbf {u}}^{(k-1)} ||_{L^2_{\nu }(\varGamma ) \otimes {\mathbb {R}}^N_{\mathbf {M}}} < tol\) and return \({\mathbf {u}}^{(k)}\) as the approximate eigenvector.

### Remark 2

For the computation of non-extremal eigenmodes, one may proceed as in [11] and replace \(\mathbf {K}(y)\) in (34) with \((\mathbf {K}(y) - \lambda \mathbf {M})\), where \(\lambda \in {\mathbb {R}}\) is a suitably chosen parameter. In this case we expect the algorithm to converge to an eigenpair for which the eigenvalue is close to \(\lambda \). Note, however, that now the existence of a unique solution to (34) is not necessarily guaranteed by Lemma 1.

### Algorithm 2

- (1)Solve \({\hat{{\mathbf {v}}}}= \{ v_{\alpha i} \}_{\alpha \in \mathcal {A}, i \in J} \in {\mathbb {R}}^{PN}\) from the linear system$$\begin{aligned} \widehat{\mathbf {K}} {\hat{{\mathbf {v}}}}= \widehat{\mathbf {M}} {\hat{{\mathbf {u}}}}^{(k-1)}. \end{aligned}$$(41)
- (2)Solve \({\hat{s}} = \{ s_{\alpha } \}_{\alpha \in \mathcal {A}} \in {\mathbb {R}}^P\) from the nonlinear systemwith the initial guess \(s_{\alpha } = || {\hat{{\mathbf {v}}}}||_{{\mathbb {R}}^P \otimes {\mathbb {R}}^N_{\mathbf {M}}} \delta _{\alpha 0}\) for \(\alpha \in \mathcal {A}\).$$\begin{aligned} F({\hat{s}},{\hat{{\mathbf {v}}}}) = 0 \end{aligned}$$(42)
- (3)Solve \({\hat{{\mathbf {u}}}}^{(k)} = \{ u_{\alpha i}^{(k)} \}_{\alpha \in \mathcal {A}, i \in J} \in {\mathbb {R}}^{PN}\) from the linear system$$\begin{aligned} \mathbf {T}({\hat{s}}) {\hat{{\mathbf {u}}}}^{(k)} = {\hat{{\mathbf {v}}}}. \end{aligned}$$(43)
- (4)
Stop if \(|| {\hat{{\mathbf {u}}}}^{(k)} - {\hat{{\mathbf {u}}}}^{(k-1)} ||_{{\mathbb {R}}^P \otimes {\mathbb {R}}^N_{\mathbf {M}}} < tol\) and return \({\hat{{\mathbf {u}}}}^{(k)}\) as the approximate eigenvector.

### Remark 3

In [11] Newton’s method with the initial guess \(s_{\alpha } = ||{\mathbf {v}}_{\alpha }||_{{\mathbb {R}}^N_{\mathbf {M}}}\) was suggested for the system of Eq. (42). Here the initial guess is somewhat different and corresponds to \(s_0 = || {\mathbf {v}}||_{L^2_{\nu }(\varGamma ) \otimes {\mathbb {R}}^N_{\mathbf {M}}}\) (and \(s_{\alpha } = 0\) for \(\alpha \not = 0\)).

In general it is not guaranteed that the Newton iteration for the system (42) converges to a solution. The following proposition will give some insight to the conditions under which this happens to be the case.

### Proposition 3

### Proof

This is a direct application of the Newton-Kantorovich theorem for the equation \(F(\cdot ,{\hat{{\mathbf {v}}}}) = 0\), see [13] (Theorem 6, 1.XVIII). Note that the first derivative (Jacobian) of \(F(\cdot ,{\hat{{\mathbf {v}}}})\) at \({\hat{s}}^{(0)}\) is \(2 || {\hat{{\mathbf {v}}}}||_{{\mathbb {R}}^P \otimes {\mathbb {R}}^N_{\mathbf {M}}} I_P\) and the second derivative is represented by the tensor of coefficients \(2c_{\alpha \beta \gamma }\). \(\square \)

From Proposition 3 we see that convergence of the Newton iteration is a consequence of the boundedness of the function \(F^s\), which again is ultimately determined by the structure of the multi-index set \(\mathcal {A}\).

### 5.2 Analysis of convergence

Due to a lack of general mathematical theory for multi-parametric eigenvalue problems we rely on a slightly unconventional approach in analyzing our algorithm. First of all, we restrict ourselves to asymptotic analysis since the underlying problem is nonlinear and thus hard to analyze globally. Second, we will analyze the solutions pointwise in the parameter space and deduce convergence theorems from classical eigenvalue perturbation bounds.

#### 5.2.1 Characterization of the dominant fixed point

The classical inverse iteration converges to the dominant eigenpair of the inverse matrix. In a somewhat similar fashion the spectral inverse iteration tends to converge to a certain fixed point, which we shall refer to as the dominant fixed point. Here we will establish a connection between this dominant fixed point of the spectral inverse iteration and the dominant eigenpair of the inverse of the parametric matrix under consideration. This connection is obtained by considering the fixed point as a pointwise perturbation of the eigenvalue problem of the parametric matrix.

### Lemma 2

- (i)The pair (
*s*,*w*) given by \(s = \lambda _1 - \kappa ^{-1} x^T r\) and \(w = \kappa x + X \pi \) solves the system$$\begin{aligned} \left\{ \begin{array}{l} S w = s w + r \\ ||w||^2 = 1 + \rho . \end{array} \right. \end{aligned}$$(48) - (ii)
If \(({\tilde{s}}, {\tilde{w}}) \not = (s, w)\) also solves the system (48), then \(s > {\tilde{s}}\) or \(x^T {\tilde{w}} < 0\).

- (iii)
There exists \(C > 0\) such that \(| \kappa - 1 | \le C (|\rho | + {\hat{\lambda }}^{-2} || r ||^2)\) and \(|| \pi || \le C {\hat{\lambda }}^{-1} || r ||\).

### Proof

- (i)Let \(s(\kappa ) = \lambda _1 - \kappa ^{-1} x^T r\). For any \(\kappa \ge 1/2\) we have \(|\kappa ^{-1}x^Tr| \le {\hat{\lambda }}/4\) so thatand$$\begin{aligned} \min _{2 \le i \le N} | \lambda _i - s(\kappa ) | = \min _{2 \le i \le N} | \lambda _1 - \lambda _i - \kappa ^{-1} x^T r| \ge {\hat{\lambda }} - \frac{1}{4} {\hat{\lambda }} > \frac{1}{2} {\hat{\lambda }} \end{aligned}$$(49)The function$$\begin{aligned} ||(\varLambda - s(\kappa )I)^{-1}|| \le 2 {\hat{\lambda }}^{-1}. \end{aligned}$$(50)is strictly increasing for \(\kappa \ge 1/2\) since$$\begin{aligned} f(\kappa ) = \kappa ^2 + || (\varLambda - s(\kappa )I)^{-1} X^T r ||^2 - 1- \rho \end{aligned}$$(51)One may also verify that \(f(1/2) < 0\) and \(f(2) > 0\). Thus, we may choose \(\kappa > 1/2\) such that \(f(\kappa ) = 0\). For \(w = \kappa x + X \pi \) we obtain$$\begin{aligned} \kappa ^2 f'(\kappa )&= 2\kappa ^3 + 2 x^T r || (\varLambda - s(\kappa )I)^{-\frac{3}{2}} X^T r ||^2 \nonumber \\&\ge 2 (\kappa ^3 - (2 {\hat{\lambda }}^{-1})^3 || r ||^3) \nonumber \\&> 2 (\kappa ^3 - 2^{-3}) \ge 0. \end{aligned}$$(52)so the equation \(Sw = sw + r\) is equivalent to$$\begin{aligned} Sw - sw = \kappa Sx + SX \pi - \kappa sx - s X \pi = \kappa (\lambda _1 - s) x + X (\varLambda - s I) \pi \end{aligned}$$(53)Choosing \(s = s(\kappa )\) and \(\pi = (\varLambda - s I)^{-1} X^T r\) we see that both equations are satisfied. Moreover$$\begin{aligned} \left\{ \begin{array}{l} x^T (Sw - sw - r) = \kappa (\lambda _1 - s) - x^T r = 0 \\ X^T (Sw - sw - r) = (\varLambda - s I)\pi - X^T r = 0. \end{array} \right. \end{aligned}$$(54)$$\begin{aligned} ||w||^2 = \kappa ^2 + ||\pi ||^2 = f(\kappa ) + 1 + \rho = 1 + \rho . \end{aligned}$$(55)
- (ii)Suppose \(({\tilde{s}}, {\tilde{w}})\) also solves the system (48) and write \({\tilde{w}} = {\tilde{\kappa }}x + X {\tilde{\pi }}\) for some \({\tilde{\kappa }} \in {\mathbb {R}}\) and \({\tilde{\pi }} \in {\mathbb {R}}^{N-1}\). In the nontrivial case we have \({\tilde{\kappa }} = x^T {\tilde{w}} > 0\). Assume first that \(0 \le {\tilde{\kappa }} \le 1/2\). We haveSince \(s \ge \lambda _1 - \kappa ^{-1} ||r||\), we deduce that$$\begin{aligned} {\tilde{s}}&= \frac{{\tilde{w}}^T S {\tilde{w}} - {\tilde{w}}^T r}{|| {\tilde{w}} ||^2} = \frac{\lambda _1 {\tilde{\kappa }}^2 + {\tilde{\pi }}^T \varLambda {\tilde{\pi }} - {\tilde{w}}^T r}{|| {\tilde{w}} ||^2} \le \frac{\lambda _1 {\tilde{\kappa }}^2 + \lambda _2 ||{\tilde{\pi }}||^2 + ||{\tilde{w}}|| ||r||}{|| {\tilde{w}} ||^2} \nonumber \\&= \lambda _2 + \frac{{\tilde{\kappa }}^2}{1 + \rho } {\hat{\lambda }} + \frac{||r||}{(1 + \rho )^{\frac{1}{2}}}. \end{aligned}$$(56)Now let \({\tilde{\kappa }} \ge 1/2\). If \(({\tilde{s}},{\tilde{w}})\) is to solve (48) then, as in part (i), we should have$$\begin{aligned} s - {\tilde{s}} \ge {\hat{\lambda }} - \kappa ^{-1}||r|| - \frac{{\tilde{\kappa }}^2}{1 + \rho } {\hat{\lambda }} - \frac{||r||}{(1 + \rho )^{\frac{1}{2}}}> \left( 1 - \frac{1}{4} - \frac{1}{2} - \frac{\sqrt{2}}{8} \right) {\hat{\lambda }} > 0. \end{aligned}$$(57)From the first equation we obtain \({\tilde{s}} = \lambda _1 - {\tilde{\kappa }}^{-1} x^T r\). Due to \(|{\tilde{\kappa }}^{-1} x^T r| \le {\hat{\lambda }}/4\) the matrix \((\varLambda - {\tilde{s}}I)\) is invertible so the second equation gives \({\tilde{\pi }} = (\varLambda - {\tilde{s}} I)^{-1} X^T r\). Here \({\tilde{\kappa }} \ge 1/2\) must be chosen so that \(f({\tilde{\kappa }}) = 0\) and therefore \(({\tilde{s}},{\tilde{w}}) = (s,w)\).$$\begin{aligned} \left\{ \begin{array}{l} {\tilde{\kappa }} (\lambda _1 - {\tilde{s}}) - x^T r = 0 \\ (\varLambda - {\tilde{s}} I){\tilde{\pi }} - X^T r = 0. \end{array} \right. \end{aligned}$$(58)
- (iii)From \(f(\kappa ) = 0\) and \(\kappa \ge 1/2\) we deduce thatand$$\begin{aligned} | \kappa - 1 | \le (\kappa +1)^{-1} (|\rho | + || (\varLambda - s(\kappa )I)^{-1} X^T r ||^2) \le |\rho | + 4 {\hat{\lambda }}^{-2}||r||^2 \end{aligned}$$(59)$$\begin{aligned} ||\pi || = ||(\varLambda - s(\kappa )I)^{-1} X^T r || \le 2{\hat{\lambda }}^{-1} ||r||. \end{aligned}$$(60)

Applying Lemma 2 to the system (46) pointwise for \(y \in \varGamma \) we obtain the following result.

### Proposition 4

### Proof

*s*(

*y*) can be written as

### Remark 4

Note that we have not proven the existence of a dominant fixed point of the Algorithm 1. The residuals \(\mathbf {r}\) and \(\rho \) in Proposition 4 depend on the pair \((s, {\mathbf {v}}) \in W_{\mathcal {A}}\times W_{\mathcal {A}}^N\) and hence Lemma 2 by itself is not sufficient to guarantee the existence of a dominant fixed point.

#### 5.2.2 Convergence of the dominant fixed point to a parametric eigenpair

The next step in our analysis is to bound the error between the dominant fixed point of Algorithm 1 and the dominant eigenpair of the inverse of the parametric matrix. To this end we will use the pointwise estimate obtained previously.

From Proposition 4 we may easily deduce the following result.

### Theorem 2

*C*depends only on \(s_*\), \({\hat{\lambda }}_*\), \(\mu _h^* := \sup _{y \in \varGamma } \mu _h(y)\), \(K_* = \sup _{y \in \varGamma } || \mathbf {K}^{-1}(y)||_{{\mathbb {R}}^N_{\mathbf {M}}}\), and \(M_* = || \mathbf {M}^{-1}||_{{\mathbb {R}}^N_{\mathbf {M}}}\).

### Proof

By Proposition 1 the exact eigenvalue and eigenvector of problem (9) are analytic functions of the parameter vector \(y \in \varGamma \). This suggests that the residuals on the right hand side of Eqs. (70) and (71) can be asymptotically estimated from Proposition 2.

#### 5.2.3 Convergence of the spectral inverse iteration to the dominant fixed point

The classical inverse iteration converges to the dominant eigenpair of the inverse matrix at a speed characterized by the gap between the two largest eigenvalues. Here we will establish a similar asymptotic result for the convergence of the spectral inverse iteration towards the dominant fixed point.

### Theorem 3

### Proof

Adapting Theorem 3 to the context of Algorithm 1 we obtain the following Corollary.

### Corollary 1

### Proof

Interpret Theorem 3 in the context of Algorithm 1. The bound \(\phi _{\min } \ge s_*\) is a consequence of Lemma 1. \(\square \)

#### 5.2.4 Combined error analysis

*k*:th iterate of Algorithm 1 and by \(\mu _{h, \mathcal {A}, k} := \mu ^{(k)} \in W_{\mathcal {A}}\) the associated solution to (37). Let \(u_{h, \mathcal {A}}\) and \(u_{h, \mathcal {A}, k}\) denote the functions in \(W_{\mathcal {A}}\otimes V_h\), whose coordinates are defined by the vectors \({\mathbf {u}}_{\mathcal {A}}\) and \({\mathbf {u}}^{(k)}\) respectively. The spatial, stochastic, and iteration errors may now be separated in the following sense:

### 5.3 Numerical examples

The deterministic mesh is a uniform grid of second order quadrilateral elements in all computations. The discretization in the parameter space is obtained by setting \(\tau _m := (m+1)^{\varsigma -1}\) for \(m=1,2,\ldots \) and using the multi-index sets \(\mathcal {A}_{\epsilon }\) as defined in Proposition 2. Multi-index sets of this form have been introduced in [7] and in [5] an algorithm for generating them has been suggested.

We use a matrix free formulation of the conjugate gradient method for solving the linear systems (41) and (43). The preconditioner is constructed using the mean of the parametric matrix in question [17] and as an initial guess we set the solution of the system from the previous iteration. We wish to note that in this setting only a very few iterations of the conjugate gradient method are needed at each step of the spectral inverse iteration.

#### 5.3.1 Convergence in space

*h*. This convergence for piecewise quadratic basis functions is illustrated in Fig. 2. We observe algebraic convergence rates of order 3 and 4 for the eigenfunction and eigenvalue respectively, exactly as predicted by Theorem 1. Thus, the error behaves like \(N^{-3/2}\) and \(N^{-2}\) with respect to the number of deterministic degrees of freedom.

#### 5.3.2 Convergence in the parameter domain

Interestingly we observe two well separated clusters of values in Fig. 4b. It seems that many of the multi-indices that correspond to relatively large Legendre coefficients of the eigenfunction, account only for a marginal contribution to the eigenvalue.

#### 5.3.3 Convergence of the iteration error

Keeping the number of spatial basis functions \(N = 36{,}741\) and the parameter \(\epsilon \) fixed so that \(\# \mathcal {A}_{\epsilon } = 264\), we may investigate the convergence of the solution \((\mu _{*,k},u_{*,k})\) as a function of the number of iterations *k*. This convergence is illustrated in Fig. 6. Assuming that the variation in the eigenvalues within the parameter space is small, the value \(\lambda _{1/2}\) defined in (83) may be approximated by the ratio of the two smallest eigenvalues of the problem at \(y = 0\). Thus, Fig. 6 suggests that the error behaves asymptotically like \(\lambda _{1/2}^k\), just as predicted by Corollary 1.

#### 5.3.4 Concluding remarks and comparison to sparse collocation

Using the finest levels of discretization, i.e., \(N = 9296\) degrees of freedom for approximation in space and \(\# \mathcal {A}_{\epsilon } = 121\) degrees of freedom for approximation in the parameter domain, and computing \(k = 9\) steps of the inverse iteration we obtain a solution for which the \(L^2_{\nu }(\varGamma ) \otimes L^2(D)\) error of the eigenfunction is approximately \(3 \times 10^{-6}\). The number of total degrees of freedom in this case is more than \(10^6\) and the number of active dimensions is \(M(\mathcal {A}_{\epsilon }) = 60\). The total computational time on a standard desktop machine is approximately five minutes, most of which is spent in the conjugate gradient method for the linear systems (41) and (43).

When the solution computed via the spectral inverse iteration is compared to the results of the non-composite version of the sparse collocation method introduced in [4] and employed in e.g. [1] (see equations (5.12)–(5.13) and (5.16)–(5.17)), the statistics of the two solutions seem to almost coincide. Again using the finest levels of discretization (\(N = 9296\) and \(\# \mathcal {A}_{\epsilon } = 121\)) for both methods, the \(L^2(D)\) errors of mean and variance of the eigenfunction are both less than \(3 \times 10^{-8}\) and the errors in the eigenvalue are less than \(3 \times 10^{-11}\) and \(3 \times 10^{-9}\) for the mean and variance respectively.

## 6 Spectral subspace iteration

In this section we extend the spectral inverse iteration to a spectral subspace iteration, with which we can compute dominant subspaces of the inverse of the parametric matrix under consideration. The underlying assumption is that the subspace is sufficiently smooth with respect to the parameters. Convergence of the spectral subspace iteration is verified through numerical experiments.

### 6.1 On the analyticity of finite dimensional subspaces

- (i)
each \(\mu _q(y)\) is of finite multiplicity as an eigenvalue of

*A*(*y*) for all \(y \in \varGamma \) and - (ii)
the minimum spectral gap \(\inf _{y \in \varGamma } {{\,\mathrm{dist}\,}}({\mathcal {M}}(y), \sigma (A(y)) \backslash {\mathcal {M}}(y))\) is positive.

*y*. More precisely, let \(\{ u_q(y) \}_{q = 1}^{Q'}\) be a maximal collection of linearly independent eigenfunctions corresponding to the eigenvalues \({\mathcal {M}}(y)\) for all \(y \in \varGamma \). It is not completely unreasonable to assume that \({{\,\mathrm{span}\,}}\{ u_q(y) \}_{q = 1}^{Q'}\) is analytic, in a suitable sense, as a function of the parameter vector

*y*. This assumption is the basis of our algorithm of spectral subspace iteration. For more information on the regularity of perturbed eigenvalues see [14, 15].

### 6.2 Algorithm description

As with the classical subspace iteration, the idea in the spectral version is to perform inverse iteration for a set of vectors and orthogonalize these vectors at each step. Orthogonality should here be understood in a sense that the vectors are orthogonal for all points in the parameter space \(\varGamma \). This can be approximately achieved by performing the Gram-Schmidt orthogonalization process for the vectors in the Galerkin sense, i.e., by projecting each elementary operation to the basis \(W_{\mathcal {A}}\).

Fix a finite set of multi-indices \(\mathcal {A} \subset (\mathbb {N}_0^{\infty })_c\) and let \(P= \# \mathcal {A}\). The spectral subspace iteration for the system (16) is now defined in Algorithm 3. Observe that, if the projections were precise, then the Algorithm would correspond to performing the classical subspace iteration pointwise on \(\varGamma \). Orhtogonalization of the basis vectors via the Gram-Schmidt process is achieved in step (2). We expect Algorithm 3 to converge to an approximate basis for the *Q*-dimensional invariant subspace associated to the smallest eigenvalues of the system.

### Algorithm 3

- (1)For each \(q = 1, \ldots , Q\) solve \({\mathbf {v}}^{(q)} \in W_{\mathcal {A}}^N\) from the linear equation$$\begin{aligned} P_{\mathcal {A}} \left( \mathbf {K} {\mathbf {v}}^{(q)} \right) = \mathbf {M} {\mathbf {u}}^{(k-1,q)}. \end{aligned}$$(88)
- (2)For \(q = 1, \ldots , Q\) do
- (2.1)Set$$\begin{aligned} {\mathbf {w}}^{(q)} = {\mathbf {v}}^{(q)} - \sum _{i=1}^{q-1} P_{\mathcal {A}}\left( {\mathbf {u}}^{(k,i)} P_{\mathcal {A}} \left( \langle {\mathbf {v}}^{(q)}, {\mathbf {u}}^{(k,i)} \rangle _{{\mathbb {R}}^N_{\mathbf {M}}} \right) \right) . \end{aligned}$$(89)
- (2.2)Solve \(s^{(q)} \in W_{\mathcal {A}}\) from the nonlinear equation$$\begin{aligned} P_{\mathcal {A}} \left( (s^{(q)})^2\right) = P_{\mathcal {A}} \left( || {\mathbf {w}}^{(q)} ||_{{\mathbb {R}}^N_{\mathbf {M}}}^2 \right) . \end{aligned}$$(90)
- (2.3)Solve \({\mathbf {u}}^{(k,q)} \in W_{\mathcal {A}}^N\) from the linear equation$$\begin{aligned} P_{\mathcal {A}} \left( s^{(q)} {\mathbf {u}}^{(k,q)} \right) = {\mathbf {w}}^{(q)}. \end{aligned}$$(91)

- (2.1)
- (3)
Stop if a suitable criterion is satisfied and return \(\{ {\mathbf {u}}^{(k,q)} \}_{q=1}^Q \subset W_{\mathcal {A}}^N\) as the approximate basis for the subspace.

In general we can not expect the output vectors \(\{ {\mathbf {u}}^{(k,q)} \}_{q=1}^Q \subset W_{\mathcal {A}}^N\) of Algorithm 3 to converge to any particular eigenvectors of the system (16). However, we still expect them to approximately span the subspace associated to the smallest eigenvalues of the system. In view of Sect. 6.1, if a cluster of eigenvalues is sufficiently well separated from the rest of the spectrum, then we assume the associated subspace to be analytic with respect to the parameter vector \(y \in \varGamma \). In this case we may expect optimal convergence of the projections in the Algorithm.

### Remark 5

In order to measure convergence of the Algorithm 3 we should be able to estimate the angle between subspaces over the parameter space \(\varGamma \). It is not entirely trivial to perform this kind of a computation in practise. The numerical examples in Sect. 6.3 will hopefully give some more insight on this.

### Remark 6

*y*. In this case we can modify the Algorithm 3 by adding the step

- (2.0)
Set \({\mathbf {v}}^{(1)} = \sum _{q=1}^Q {\mathbf {v}}^{(q)}\)

*y*.

Using the tensors defined in Sect. 5 we may write Algorithm 3 in the following form.

### Algorithm 4

- (1)For each \(q = 1, \ldots , Q\) solve \({\hat{{\mathbf {v}}}}^{(q)} \in {\mathbb {R}}^{PN}\) from the linear system$$\begin{aligned} \widehat{\mathbf {K}} {\hat{{\mathbf {v}}}}^{(q)} = \widehat{\mathbf {M}} {\hat{{\mathbf {u}}}}^{(k-1,q)}. \end{aligned}$$(92)
- (2)For \(q = 1, \ldots , Q\) do
- (2.1)Set$$\begin{aligned} \hat{\mathbf {w}}^{(q)} = {\hat{{\mathbf {v}}}}^{(q)} - \sum _{i=1}^{q-1} \mathbf {T}\left( F^v({\hat{{\mathbf {v}}}}^{(q)},{\hat{{\mathbf {u}}}}^{(k,i)}) \right) {\hat{{\mathbf {u}}}}^{(k,i)}. \end{aligned}$$(93)
- (2.2)Solve \({\hat{s}}^{(q)} \in {\mathbb {R}}^P\) from the nonlinear systemwith the initial guess \(s^{(q)}_{\alpha } = || \hat{\mathbf {w}}^{(q)} ||_{{\mathbb {R}}^P \otimes {\mathbb {R}}^N_{\mathbf {M}}} \delta _{\alpha 0}\) for \(\alpha \in \mathcal {A}\).$$\begin{aligned} F({\hat{s}}^{(q)},\hat{\mathbf {w}}^{(q)}) = 0 \end{aligned}$$(94)
- (2.3)Solve \({\hat{{\mathbf {u}}}}^{(k,q)} \in {\mathbb {R}}^{PN}\) from the linear system$$\begin{aligned} \mathbf {T}({\hat{s}}^{(q)}) {\hat{{\mathbf {u}}}}^{(k,q)} = \hat{\mathbf {w}}^{(q)}. \end{aligned}$$(95)

- (2.1)
- (3)
Stop if a suitable criterion is satisfied and return \(\{ {\hat{{\mathbf {u}}}}^{(k,q)} \}_{q=1}^Q \subset {\mathbb {R}}^{PN}\) as the approximate basis for the subspace.

### 6.3 Numerical examples

*y*.

*Q*-smallest eigenvalues of the problem. We define

## 7 Conclusions and future prospects

We have presented a comprehensive error analysis for the spectral inverse iteration, when applied to solving the ground state of a stochastic elliptic operator. We have also proposed a method of spectral subspace iteration and, using numerical examples, shown its potential in computing approximate subspaces associated to possibly clustered eigenvalues. Further analysis, both numerical and theoretical, of this algorithm is left for future research.

The numerical examples suggest that our algorithms are both accurate and efficient. However, theoretical estimates for the computational complexity are not entirely trivial to obtain as this would require information on the structure of the tensor of coeffiecients \(c_{\alpha \beta \gamma }\). Moreover, when iterative solvers are used, the optimal strategy is to increase the associated tolerances in the course of the iteration. We note that sparse products of the spatial and stochastic approximation spaces, as in [5], may be applied to further reduce the computational effort, and that matrix free algorithms also allow for easy parallelization.

## Notes

### Acknowledgements

Open access funding provided by Aalto University.

## References

- 1.Andreev, R., Schwab, C.: Sparse tensor approximation of parametric eigenvalue problems. In: Lecture notes in computational science and engineering, vol. 83, pp. 203–241. Springer, Berlin (2012)Google Scholar
- 2.Babuška, I., Nobile, F., Tempone, R.: A stochastic collocation method for elliptic partial differential equations with random input data. SIAM J. Numer. Anal.
**45**(3), 1005–1034 (2007)MathSciNetCrossRefzbMATHGoogle Scholar - 3.Babuška, I., Osborn, J.: Eigenvalue problems. In: Handbook of Numerical Analysis, vol. II, pp. 641–787. Elsevier Science Publishers B.V., North-Holland (1991)Google Scholar
- 4.Bieri, M.: A sparse composite collocation finite element method for elliptic SPDEs. SIAM J. Numer. Anal.
**49**(6), 2277–2301 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 5.Bieri, M., Andreev, R., Schwab, C.: Sparse tensor discretization of elliptic SPDEs. SIAM J. Sci. Comput.
**31**(6), 4281–4304 (2009)MathSciNetCrossRefzbMATHGoogle Scholar - 6.Bieri, M., Andreev, R., Schwab, C.: Sparse tensor discretization of elliptic spdes. Tech. Rep. 2009-07, Seminar for Applied Mathematics, ETH Zürich, Switzerland. https://www.sam.math.ethz.ch/sam_reports/reports_final/reports2009/2009-07.pdf (2009)
- 7.Bieri, M., Schwab, C.: Sparse high order FEM for elliptic sPDEs. Comput. Methods Appl. Mech. Eng.
**198**, 1149–1170 (2009)MathSciNetCrossRefzbMATHGoogle Scholar - 8.Boffi, D.: Finite element approximation of eigenvalue problems. Acta Numer.
**19**, 1–120 (2010)MathSciNetCrossRefzbMATHGoogle Scholar - 9.Ghanem, R., Spanos, P.: Stochastic Finite Elements: A Spectral Approach. Dover Publications, Inc., Mineola (2003)zbMATHGoogle Scholar
- 10.Gunawan, H., Neswan, O., Setya-Budhi, W.: A fromula for angles between subspaces of inner product spaces. Contrib. Algebra Geom.
**46**(2), 311–320 (2005)zbMATHGoogle Scholar - 11.Hakula, H., Kaarnioja, V., Laaksonen, M.: Approximate methods for stochastic eigenvalue problems. Appl. Math. Comput.
**267**(C), 664–681 (2015). https://doi.org/10.1016/j.amc.2014.12.112 MathSciNetzbMATHGoogle Scholar - 12.Henrot, A.: Extremum Problems for Eigenvalues of Elliptic Operators. Birkhäuser, Basel (2006)CrossRefzbMATHGoogle Scholar
- 13.Kantorovich, L., Akilov, G.: Functional Analysis in Normed Spaces. Pergamon Press, New York (1964)zbMATHGoogle Scholar
- 14.Kato, T.: Perturbation Theory for Linear Operators. Springer, Berlin (1997)Google Scholar
- 15.Kriegl, A., Michor, P., Rainer, A.: Denjoy-carleman differentiable perturbation of polynomials and unbounded operators. Integr. Equ. Oper. Theory
**71**, 407–416 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 16.Meidani, H., Ghanem, R.: Spectral power iterations for the random eigenvalue problem. AIAA J.
**52**, 912–925 (2014)CrossRefGoogle Scholar - 17.Powell, C.E., Elman, H.C.: Block-diagonal preconditioning for spectral stochastic finite-element systems. IMA J. Numer. Anal.
**29**(2), 350–375 (2008)MathSciNetCrossRefzbMATHGoogle Scholar - 18.Soize, C., Ghanem, R.: Physical systems with random uncertainties: chaos representations with arbitrary probability measure. SIAM J. Sci. Comput.
**26**, 395–410 (2004)MathSciNetCrossRefzbMATHGoogle Scholar - 19.Sousedík, B., Elman, H.C.: Inverse subspace iteration for spectral stochastic finite element methods. SIAM/ASA J. Uncertain. Quantif.
**4**, 163–189 (2016)MathSciNetCrossRefzbMATHGoogle Scholar - 20.Verhoosel, C.V., Gutiérrez, M.A., Hulshoff, S.J.: Iterative solution of the random eigenvalue problem with application to spectral stochastic finite element systems. Int. J. Numer. Methods Eng.
**68**, 401–424 (2006)MathSciNetCrossRefzbMATHGoogle Scholar - 21.Xiu, D., Karniadakis, G.E.: The Wiener–Askey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput.
**24**, 619–644 (2002)MathSciNetCrossRefzbMATHGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.