On restarting the tensor infinite Arnoldi method
 558 Downloads
Abstract
An efficient and robust restart strategy is important for any Krylovbased method for eigenvalue problems. The tensor infinite Arnoldi method (TIAR) is a Krylovbased method for solving nonlinear eigenvalue problems (NEPs). This method can be interpreted as an Arnoldi method applied to a linear and infinite dimensional eigenvalue problem where the Krylov basis consists of polynomials. We propose new restart techniques for TIAR and analyze efficiency and robustness. More precisely, we consider an extension of TIAR which corresponds to generating the Krylov space using not only polynomials, but also structured functions, which are sums of exponentials and polynomials, while maintaining a memory efficient tensor representation. We propose two restarting strategies, both derived from the specific structure of the infinite dimensional Arnoldi factorization. One restarting strategy, which we call semiexplicit TIAR restart, provides the possibility to carry out locking in a compact way. The other strategy, which we call implicit TIAR restart, is based on the Krylov–Schur restart method for the linear eigenvalue problem and preserves its robustness. Both restarting strategies involve approximations of the tensor structured factorization in order to reduce the complexity and the required memory resources. We bound the error introduced by some of the approximations in the infinite dimensional Arnoldi factorization showing that those approximations do not substantially influence the robustness of the restart approach. We illustrate the effectiveness of the approaches by applying them to solve large scale NEPs that arise from a delay differential equation and a wave propagation problem. The advantages in comparison to other restart methods are also illustrated.
Keywords
Nonlinear eigenvalue problem Restart Tensor infinite Arnoldi Krylov subspace method Krylov–Schur methodMathematics Subject Classification
35P30 65H17 65F60 15A18 65F151 Introduction
There is a large number of methods available in a large amount of numerical linear algebra literature for (1). There are specialized methods for solving different classes of NEPs such as polynomial eigenvalue problems (PEPs) see [18, 22, 23] and [2, Chapter 9], in particular quadratic eigenvalue problems (QEPs) [3, 24, 25, 33] and rational eigenvalue problems (REPs) [5, 6, 30, 36]. There are also methods that exploit the structure of the operator \(M(\lambda )\) like Hermitian structure [31, 32] or low rank of the matrixcoefficients [34]. Methods for solving a more general class of NEP are also present in literature. There exist methods based on modification of the Arnoldi method [37], which can be restarted for certain problems, Jacobi–Davidson methods [8], Newtonlike methods [9, 16, 28]. Finally, there is a class of methods (to which the presented method belongs) based on Krylov methods and rational Krylov methods that can be interpreted as either dynamically expanding an approximation of the NEP or applying a method on an infinite dimensional operator [11, 14, 35].
A problematic aspect of any algorithm based on the Arnoldi method is that, when many iterations are performed, the computation time per iteration will eventually become large. Moreover, finite arithmetic aspects may restrict the accuracy. Fortunately, an appropriate restart of the algorithm can resolve these issues in many situations. There exist two main classes of restarting strategies: explicit restart and implicit restart. Most of the explicit restart techniques correspond to selecting a starting vector that generates an Arnoldi factorization with the wanted Ritz values. The implicit restart consists in computing a new Arnoldi factorization, without explicitly computing a starting vector, with the wanted Ritz values. This process can be done deflating the unwanted Ritz values as in, e.g., IRA [20] or extracting a proper subspace of the Krylov space by using the Krylov–Schur restart approach [29]. For reasons of numerical stability, the implicit restart is often considered more robust than explicit restart. See [27] for further discussions about the restart of the Arnoldi method for the linear eigenvalue problem.

An implicit restart, which consists in an adaption of the Krylov–Schur restart,

A semiexplicit restart, which consists in an explicit restart by imposing the structure on the converged locked Ritz pairs and on the starting function.
A semiexplicit restart for IAR was presented in [12]. We extend the procedure to TIAR. The feature of imposing the structure on the converged Ritz values and starting function is obtained by generating the Krylov space using a particular type of structured functions, which are sums of polynomial and exponential functions. We show that such functions can be included in the framework of TIAR. In particular we carry out a memory efficient representation of the structured functions, similar to [13].
There exist other Arnoldilike methods combined with a companion linearization that use memory efficient representation of the Krylov basis matrix and that can be restarted. There are, for instance, TOAR [17, 39] and CORK [35], which are based on the concept of compact Arnoldi decompositions [21]. Similar to TIAR, the direct usage of the Krylov–Schur restart for these methods does not decrease the complexity unless SVDbased approximations are used (which is indeed suggested in the implementation of the methods). More precisely, the coefficients that represent the Krylov basis are replaced with their low rank approximations. In contrast to those approaches, our specific setting, in particular the infinite dimensional function formulation and the representation of the basis with tensor structure functions, allows us to characterize the impact of the approximations.
The relationship with CORK [35] can be seen as follows. A variation of a special case of our approach (implicit restart without SVDcompression and without polynomial degree reduction) has similarities with a special case of a variation of CORK (single shift, with a particular companion linearization without SVDcompression). Our approach is based on a derivation involving infinite dimensionality which allows us to derive theory for the truncation and it allows us to restart with infinite dimensional objects. This strategy is effective since the invariant pairs are infinite dimensional objects. In contrast to this, CORK is derived from reasoning concerning the NEP linearization. This allows the usage of different types of companion linearizations, that correspond to different approximations of the nonlinearities of \(M(\lambda )\), and leads to a rational Krylov approach, i.e., several shifts can be used in one run.
The paper is organized as follows: in Sect. 2 we extend TIAR to tensor structured functions. In Sect. 3 we present a derivation of the Krylov–Schur type restarting in an abstract and infinite dimensional setting. Section 4 contains the derivation of a semiexplicit restart for TIAR. In Sect. 5 we carry out the adaption of Krylov–Schur restart for TIAR. We analyze the complexity of the proposed methods in Sect. 6. Finally, in Sect. 7 we show the effectiveness of the restarting strategies with numerical simulations to large and sparse NEPs.
We denote by \(a_{:,:,:}\) a threedimensional tensor and by \(a_{i,:,:}, a_{:,j,:}\) and \(a_{:,:,\ell }\) the slices of the tensor with respect to the first, second and third dimension. The vector \(z_j\) denotes the j–th column of the matrix Z and \(e_j\) the j–th canonical unit vector. The matrix \(I_{m,p}\) denotes the matrix obtained by extracting the fist m rows and p columns of a larger square identity matrix. The matrix \(H_k\) denotes the square matrix obtained by removing the last row from the matrix \(\underline{H}_k \in \mathbb {C}^{(k+1) \times k }\).
2 Tensor structured functions and TIAR factorizations
Our main algorithms are derived using particular types of functions. More precisely, we consider functions that can be expressed as \(\psi (\theta ) = q(\theta ) + Y \exp (S \theta ) c\) where \(q: \mathbb {C}\rightarrow \mathbb {C}^n\) is a polynomial, \(Y \in \mathbb {C}^{n \times p}, S \in \mathbb {C}^{p \times p}\), \(c \in \mathbb {C}^{p}\) and \(\exp (S \theta )\) denotes the matrix exponential. Such functions were also used in [12]. We now introduce a new memoryefficient representation of such functions involving tensors.
Definition 1
Similar to many restart strategies for linear eigenvalue problems, our approach is based on computation, representation and manipulation of an Arnolditype factorization. For our infinite dimensional operator, the analogous Arnolditype factorization is defined as follows.
Definition 2
2.1 Action of \(\mathscr {B}\) on tensor structured functions
Theorem 3
Proof
Remark 4
The assumption (10) can only be satisfied if \(r+p \le n\). This is the generic case that we consider in this paper. Our focus is on large–scale NEPs and, in Sect. 5.1, we introduce approximations that avoid r from being large. The hypothesis \(\lambda (S) \subseteq \varOmega \) is necessary in order to define \(\mathbb {M}_d(Y,S)\) that is used to compute \({\tilde{z}}\) in Eq. (9).
2.2 Orthogonality
The tensor structured functions in the TIAR factorization (Definition 2) are orthonormal. In order to impose the orthogonality in our algorithms, we now present a theory which characterizes the orthonormality of tensor structured functions in terms of their coefficients. In particular, we derive the theory necessary to carry out the Gram–Schmidt orthogonalization. Since most of the orthogonalization procedures involve linear combinations of vectors, we start with the observation that linear combinations of tensor structured functions carry over directly to the coefficients.
Observation 5
Theorem 6
Proof
Let us define \(h_j:=\langle \psi , \psi _j\rangle \) for \(j=1, \dots , k\). The orthogonal complement, computed with the Gram–Schmidt process, is \(\psi ^\perp (\theta ) = \psi (\theta )  \varPsi _k(\theta ) h\). Using the Observation 5 we obtain directly (18).
3 Restarting for TIAR in an abstract setting
3.1 A TIAR expansion algorithm in finite dimension
One algorithmic component common in many restart procedures is the expansion of an Arnolditype factorization. The standard way to expand Arnolditype factorizations (as, e.g., described in [29, Sect. 3]) requires the computation of the action of the operator/matrix and orthogonalization. We now show how we can carry out an expansion of the infinite dimensional TIAR factorization (7) by only using operations on matrices and vectors (of finite dimension).
3.2 The Krylov–Schur decomposition for TIAR factorizations
Observation 7
In the TIAR factorization (27), \(({\bar{\varPsi }}_{p_\ell }, R_{1,1})\) is an approximation of an invariant pair, i.e., \(\mathscr {B}{\bar{\varPsi }}_{p_\ell } = {\bar{\varPsi }}_{p_\ell } R_{1,1}\). Moreover \(({\bar{\varPsi }}_{p_\ell }(0), R_{1,1}^{1})\) is an approximation of an invariant pair of the original NEP in the sense of [16, Definition 1], see [12, Theorem 2.2].
3.3 Two structured restarting approaches
The standard restart approach for TIAR using Krylov–Schur type restarting, as described in the previous section, involves expansions and manipulations of the TIAR factorization. Due to the linearity of tensor structured functions with respect to the coefficients, described in Observation 5, the manipulations for \(\varPsi _{m}\) leading to \(\varPsi _p\) can be directly carried out on the coefficients representing \(\varPsi _m\). Unfortunately, due to the implicit representation of \(\varPsi _m\), the memory requirements are not substantially reduced since the basis matrix \(Z\in \mathbb {C}^{n\times r}\) is not modified in the manipulations. The size of the basis matrix Z is the same before and after the restart.

Semiexplicit restart (Sect. 4): An invariant pair can be completely represented by exponentials and therefore it does not contribute to the memory requirement for Z. The fact that invariant pairs are exponentials was exploited in the restart in [12]. We show how the ideas in [12] can be carried over to tensor structured functions. More precisely, the adaption of [12] involves restarting the iteration with a locked pair, i.e., only the first \(p_\ell \) columns of (27), and a function f constructed in a particular way. The approach is outlined in Algorithm 2 with details specified in Sect. 4.

Implicit restart (Sect. 5): By only representing polynomials, we show that the TIAR factorization has a particular structure such that it can be accurately approximated. This allows us to carry out a full implicit restart, and subsequently approximate the TIAR factorization reducing the size of the matrix Z. The adaption is given in Algorithm 3. The approximation of the TIAR factorization in Step 6 is specified in Algorithm 4 and derived in Sect. 5, including an error analysis.
4 Tensor structure exploitation for the semiexplicit restart
The IAR restart approach in [12] is based on representing functions as sums of exponentials and polynomials. An attractive feature of that approach is that the invariant pairs can be exactly represented and locking can be efficiently incorporated. Due to the explicit storage of polynomial coefficients, the approach still requires considerable memory. We here show that, by representing the functions implicitly as a tensor structured functions (3), we can maintain all the advantages but improve performance (both in memory and CPUtime). This construction is equivalent to [12], but more efficient.
The expansion of the TIAR factorization with tensor structured functions (as described in Algorithm 1), combined with the locking procedure (as described in Sect. 3.2), and imposing the structure to the invariant pair as in [12], results in Algorithm 2. Steps 3–10 follow the procedure described in [12] adapted for tensor structured functions. In particular Steps 3– 6 consist in extracting and imposing the structure on the invariant pair \(({\bar{\varPsi }}, R_{1,1})\). In Steps 7–9 a new starting function f is selected and orthogonalized with respect to \({\bar{\varPsi }}\) and in Step 10 the new TIAR factorization is defined.
5 Tensor structure exploitation for the implicit polynomial restart
In contrast to the procedure in Sect. 4, where the main idea was to do locking with exponentials and restart with a factorization of length \(p_\ell \), we now propose a fully implicit procedure involving a factorization of length p. In this setting we use \(C=0\), i.e., we only represent polynomials with the tensor structured functions. This allows us to derive theory for the structure of the coefficient matrix, which shows how to approximate the TIAR factorization. This procedure is summarized in Algorithm 3.
The approximation in Step 6 is done in order to reduce the growth in memory requirements for the representation. The approximation technique is derived in the following subsections and summarized in Algorithm 4.
Our approximation approach is based on an approximation with a truncated singular value decomposition and a degree reduction. A compression with a truncated singular value decomposition was also made for the compact representations in CORK [35] and TOAR [17]. Our specific setting allows us to prove bounds on the error introduced by the approximations (Sects. 5.1, 5.2). We also show the effectiveness by proving a bound on the decay of the singular values (Sect. 5.3).
Theorem 8
Proof
Let \(L_k^\dagger \) denote the pseudoinverse of \(L_k\). In order to show (31), we now show that \(\Vert R^{1} \Vert _F =\Vert L_k^\dagger \Vert _F\) and \(\kappa (R) = \kappa (L_k)\) where \(\kappa (L_k)=\Vert L_k\Vert _F\Vert L_k^\dagger \Vert _F\). Due to the equivalence of TIAR and IAR and the companion matrix interpretation of IAR [14, theorem 6], we have that TIAR is equivalent to using the Arnoldi method on the matrix \(C_{k+1}\) with starting vector \(v=\sum _{\ell =1}^r {\bar{a}}_{:,\ell } z_\ell \). More precisely, the relation \(\Phi _{k+1} = \varPsi _{k+1} R\) can be written in terms of vectors as \(L_{k+1} = V_{k+1} R\) where the first column of \(V_{k+1}\) and \(L_{k+1}\) is \(v=\sum _{\ell =1}^r {\bar{a}}_{:,\ell } \otimes z_\ell \) and \(L_k=[v, C_{k+1} v, \dots , C_{k+1}^{k+1}v]\). By using the orthogonality of \(V_{k+1}\) we conclude that that \(\Vert R^{1} \Vert _F =\Vert L_k^\dagger \Vert _F\) and \(\kappa (R) =\kappa (L_k)\).
The approximations that we introduce in the next sections (required in Step 6 of Algorithm 3) are based on the assumption that the tensor structured functions \(\varPsi ^{(j)}\) are such that the decay constant \(C(\varPsi ^{(j)})\) is small. Theorem 8 shows that this constant remains small after the TIAR expansion in Step 2 if \(\kappa (L_k)\) is not large. However, the condition number of the Krylov matrix \(L_k\) can be large, see [4]. This does not necessarily imply that the decay constant is large. Notice that if \((\varPsi _k, H_k)\) is an invariant pair, \(L_k\) has linearly dependent columns and \(\kappa (L_k)\) is infinite. Analogously, if \((\varPsi _k, H_k)\) is (in this sense) close to an invariant pair, we expect \(\kappa (L_k)\) to be large. Hence, in these situations, the righthand side of (31) is expected to be large. However, the decay constant is not expected to be large, since the decay constant for an invariant pair is given by (30). Note that the decay is also preserved in the operations associated with the restart. After the Ritzvalue selection (Step 3–4) the new TIAR factorization is computed in Step 5. Since \(\varPsi ^{(j+1)}\) is obtained through a unitary transformation from \(\varPsi ^{(j)}\), by using the properties of the Frobenius norm, we get \(C(\varPsi ^{(j+1)}) \le \sqrt{p+1} C(\varPsi ^{(j)})\).
5.1 Approximation by SVD compression
Given a TIAR factorization with basis function \(\varPsi _k\) we now show (in the following theorem) how we can approximate the basis function with less memory, by using a thinner basis matrix \(Z\in \mathbb {C}^{n\times \tilde{r}}\) where \(\tilde{r}\) should be selected as small as possible. The theorem also shows how this approximation influences the approximation \(\varPsi _k\) as well as the residual of the TIAR factorization. It turns out that the residual error is small if \(\sigma _{\tilde{r}}\) is small, where \(\sigma _1,\ldots ,\sigma _r\) are singular values associated with the coefficient tensor. This implies that \(\tilde{r}\) can be chosen small if we have a fast singular value decay. We characterize the decay of the singular values in Sect. 5.3.
Theorem 9
Proof
5.2 Approximation by reducing the degree
Another approximation which reduces the complexity can be done by truncating the polynomial in \(\varPsi _k\). The following theorem illustrates the approximation properties of this approach.
Theorem 10
Proof
By definition \( \varPsi _k(\theta ) = \varPsi _{k+1}(\theta ) I_{k+1,k} \) and \( {\tilde{\varPsi }}_k(\theta ) = {\tilde{\varPsi }}_{k+1}(\theta ) I_{k+1,k} \), using the Observation 5, if we define \(Y_i := X_i I_{k+1,k}\) for \(i=0, \dots , d1\) and \(Y:=[ Y_0^H \dots Y_{d1}^H ]^H\) and \({\tilde{Y}}:=[ Y_0^H \dots {\tilde{Y}}_{{\tilde{d}}  1}^H ]^H\) we can express \( \varPsi _k(\theta ) = P_{d1}(\theta ) Y \) and \( {\tilde{\varPsi }}_k(\theta ) = P_{{\tilde{d}}1}(\theta ) {\tilde{Y}}. \)
Remark 11
The approximation given in Theorem 10 can only be effective under the condition that \( \left( \max _{{\tilde{d}}+1 \le i \le d} \Vert M_i \Vert _F \right) /({\tilde{d}}+1)!\) is small. In particular this condition is satisfied if the Taylor coefficients \( M_i /i!\) present a fast decay. This condition corresponds to having the coefficients of the power series expansion of \(M(\lambda )\) that are decaying to zero.
The final approximation procedure is summarized in Algorithm 4. In particular Step 1–2 correspond to the approximation by SVD compression, described in Sect. 5.1, whereas Step 3–4 correspond to the approximation by reducing the degree, described in Sect. 5.2.
5.3 The fast decay of singular values
Finally, as a further justification for our approximation procedure, we now show how fast the singular values decay. The fast decay in the singular values illustrated below justifies the effectiveness of the truncation in Sect. 5.1.
Lemma 12
Let \( a \in \mathbb {C}^{(d+1) \times (k+1) \times r}, Z \in \mathbb {C}^{n \times r}\) be the coefficients and the matrix that represent the tensor structured function \(\varPsi _{k+1}\) and let \(\underline{H}_{k} \in \mathbb {C}^{(k+1) \times k}\) be such that \((\varPsi _{k+1}, \underline{H}_{k})\) is a TIAR factorization. Then, the tensor a is generated by \(d+1\) vectors, in the sense that each vector \(a_{i,j,:}\) for \(i=1, \dots , d+1\) and \(j=1, \dots , k+1\) can be expressed as linear combination of the vectors \( a_{i,1,:}\) and \(a_{1,j,:}\) for \( i = 1, \dots , dk \) and \( j = 1, \dots , k+1\).
Proof
The proof is based on induction over the length k of the TIAR factorization. The result is trivial if \(k=1\). Suppose the result holds for some k. Let \(Z \in \mathbb {C}^{n \times (r1)}, a \in \mathbb {C}^{d \times k \times (r1)}\) represent the tensor structured function \(\varPsi _{k}\) and let \(\underline{H}_{k1} \in \mathbb {C}^{k \times (k1)}\) be an upper Hessenberg matrix such that \((\varPsi _k,\underline{H}_{k1})\) is a TIAR factorization. If we expand the TIAR factorization \((\varPsi _k,\underline{H}_{k1})\) by using the Algorithm 1, more precisely by using (13b) and (18b), we obtain \(\beta a_{i+1,k+1,:} = a_{i,k,:}/i  \sum _{j=1}^k h_j a_{i,j,:}\) for \(i=1, \dots , d\). We reach the conclusion by induction.
Theorem 13
Proof
6 Complexity analysis
We presented above two different restarting strategies: the structured semiexplicit restart and the implicit restart. They have different performance for different problems and we have not been able to conclusively determine if one is better than the other. The best choice of the restarting strategy appears to depend on many problem properties. It may be convenient to test both methods on the same problem. We now discuss the general performances, in terms of complexity and stability. The complexity discussion is based on the assumption that the complexity of the action of \(M_0^{1}\) is neglectable in comparison to the other parts of the algorithm.
6.1 Complexity of expanding the TIAR factorization
The main computational effort of the Algorithm 2 and Algorithm 3 is the expansion of a TIAR factorization described in Algorithm 1, independent of which restarting strategy is used. The essential computational effort of Algorithm 1 is the computation of \({\tilde{z}}\), given in Eq. (9). This operation has complexity \(\mathscr {O}(drn)\) for each iteration. In the implicit restart, Algorithm 3, r and d are not large in general, due to the way they are automatically selected in the approximation procedure in Algorithm 4. In the semi–explicit restart, Algorithm 2, we have instead \(r, d \le m\).
6.2 Complexity of the restarting strategies
After an implicit restart we obtain a TIAR factorization of length p, whereas after a semiexplicit restart, we obtain a TIAR factorization of length \(p_\ell \). This means that the semi–explicit restart requires a re–computation phase, i.e., after the restart we need to perform extra \(pp_\ell \) steps in order to have a TIAR factorization of length p. If \(pp_\ell \) is large, i.e., if not many Ritz values converged in comparison to the restarting parameter p, then the recomputation phase is the essential computational effort of the algorithm. Notice that this is hard to predict since we do not know how fast the Ritz values will converge in advance.
6.3 Stability of the restarting strategies
We will illustrate in Sect. 7 that the restarting approaches have different stability properties. The semiexplicit restart tends to be efficient if only a few eigenvalues are wanted, i.e., if \(p_\mathrm{max}\) is small. This is due to the fact that we impose the structure in the starting function. On the other hand, the implicit restart requires a thick restart in order to be stable in several situations, see corresponding discussions for the linear case in [19, chapter 8]. Then p has to be large enough in a sense that at each restart the p wanted Ritz values have the corresponding residual not small. This leads to additional computational and memory resources.
If we use the semiexplicit restart, then the computation of \({\tilde{z}}\), in Eq. (9), involves the term \(\mathbb {M}_d(Y,S)\). This quantity can be computed in different ways. In the simulations we must choose between (8) or [12, Equation (4.8)]. The choice influences the stability of the algorithm. In particular if one eigenvalue of S is close to \(\partial \varOmega \) and \(M(\lambda )\) is not analytic on \(\partial \varOmega \), the series (8) may converge slowly and in practice overflow can occur. In such situations, [12, Equation (4.8)] is preferable. Notice that it is not always possible to use [12, Equation (4.8)] since many problems cannot be formulated as a short sum of product of matrices and functions.
6.4 Memory requirements of the restarting strategies
From a memory point of view, the essential part of the semiexplicit restart is the storage of the matrices Z and Y, that is \(\mathscr {O}(nm+np)\). In the implicit restart the essential part is the storage of the matrix Z and requires \(\mathscr {O}(nr_{\max })\) where \(r_{\max }\) denotes the maximum value that the variable r takes in the Algorithm 1. The size of \(r_{\max }\) is not predictable since it depends on the SVDapproximation introduced in Algorithm 4. Since in each iteration of the Algorithm 1 the variable r is increased, it holds \(r_{\max } \ge mp\). Therefore, in the optimal case where \(r_{\max }\) takes the lower value, the two methods are comparable in terms of memory requirements. Notice that, the semi–explicit restart in general requires less memory and has the advantage that the required memory is problem independent.
7 Numerical experiments
7.1 Delay eigenvalue problem
In order to illustrate properties of the proposed restart methods and advantages in comparison to other approaches, we carried out numerical simulations^{1} for solving a delay eigenvalue problem (DEP). More precisely, we consider the DEP obtained by discretizing the characteristic equation of the delay differential equation defined in [15, sect 4.2, Eq (22a)] with \(\tau =1\). By using a standard second order finite difference discretization, the DEP is formulated as \(M(\lambda ) =  \lambda ^2 I + \lambda T_1 + T_0 + e^{\lambda } T_2\). We show how the proposed methods perform in terms of m, the maximum length of the TIAR factorization, and the restarting parameter p.
Table 1a, b show the advantages of our semiexplicit restart approach in comparison to the equivalent method described in [12]. Our new approach is faster in terms of CPUtime and can solve larger problems due to the memory efficient representation of the Krylov basis.
DEP, Semiexplicit restart
Size  Tensor struct. Functions  Original approach [12]  

CPU  Memory  CPU  Memory  
(a) \(m=20, p=5\), restart = 7  
10,201  19.07 s  3.7 MB  31.41 s  65.38 MB 
40,401  30.14 s  14.8 MB  1 m 30 s  258.92 MB 
160,801  1 m 47 s  58.9 MB  6 m 04  1.01 GB 
641,601  7 m 30 s  235.0 MB  24 m 27 s  4.02 GB 
1,002,001  12 m 01 s  366.9 MB  –  – 
(b) \(m=40, p=10\), restart = 4  
10,201  13.47 s  7.62 MB  1 m 05 s  255.27 MB 
40,401  41.81 s  30.20 MB  4 m  1 GB 
160,801  144.79 s  120.23 MB  15 m 54 s  3.93 GB 
641,601  10 m 43 s  479.71 MB  –  – 
1,002,001  16 m 21 s  749.18 MB  –  – 
7.2 Waveguide eigenvalue problem
DEP, Implicit restart
Size  Approximation  No approximation  

CPU  Memory  CPU  Memory  
(a) \(m=20, p=5\), restart = 7  
10,201  6.82 s  7.8 MB  11.95 s  17.1 MB 
40,401  21.96 s  30.8 MB  37.63 s  67.8 MB 
160,801  1 m20 s  120.2 MB  2 m21 s  269.9 MB 
641,601  5 m24 s  469.9 MB  9 m33 s  1.1 GB 
1,002,001  8 m36 s  733.9 MB  15 m16 s  1.6 GB 
(b) \(m=40, p=10\), restart = 4  
10,201  9.54 s  11.1 MB  16.61 s  20.2 MB 
40,401  30.48 s  43.8 MB  50.66 s  80.1 MB 
160,801  1 m54 s  174.2 MB  3 m11 s  319.0 MB 
641,601  8 m05 s  695.1 MB  13 m14 s  1.2 GB 
1,002,001  12 m17 s  1.1 GB  20 m57 s  1.9 GB 
DEP, Implicit and semiexplicit restart, stopped when residual of \(p_\mathrm{max}=10\) Ritz values is less than \(10^{10}\)
Size  Implicit restart  Semiexplicit restart  

Nr. restarts  CPU  Memory  Nr. restarts  CPU  Memory  
40,401  6  13.8 s  29.59 MB  7  28.97 s  14.79 MB 
160,801  4  37.02 s  115.32 MB  6  1 m 29 s  58.89 MB 
641,601  4  2 m 31 s  450.34 MB  6  6 m 23 s  234.96 MB 
1,002,001  4  3 m 58 s  703.30 MB  5  8 m 29 s  366.94 MB 
2,563,201  –  –  –  5  21 m10 s  938.67 MB 
In analogy to the previous subsection, we carried out numerical simulations in order to compare the semiexplicit and the implicit restart. With Figs. 4 and 5, we illustrate the performance of the two restarting approaches with respect to the choice of the parameters m and p. When p is sufficiently large, the residual in the semiexplicit restart appears to stagnate after the first restart whereas it decreases in a regular way in the implicit restart. See Fig. 4. This is due to the fact that semiexplicit restart imposes the structure on p vectors which is not beneficial when they do not contain eigenvector approximations. On the other hand, when p is small, the behavior of the residual appear to be specular. See Fig. 5. This is a consequence of the fact that, already after the first restart, the Krylov subspace is almost an invariant subspace (since p Ritz pairs are quite accurate). This is consistent with the linear case where implicit restarting with a Krylov subspace which is almost an invariant subspace is known to suffer from numerical instabilities. It is known that this specific problem has two eigenvalues. Therefore, in order to reduce the CPUtime and the memory resources, the restarting parameter p should be selected small. As consequence of the above discussion, we conclude that the semiexplicit restart is the best restarting strategy for this problem.
8 Concluding remarks and outlook
In this work we have derived an extension of the TIAR algorithm and two restarting strategies. Both restarting strategies are based on approximating the TIAR factorization. In other works on the IARmethod it has been proven that the basis matrix contains a structure that allows exploitations, e.g. for NEPs with low rank structure in the coefficients [34]. An investigation about the combination of the approximations of the TIAR factorization with such structures of the NEP seems possible but deserve further attention.
Although the framework of TIAR and restarted TIAR is general, a specialization of the methods to the NEP is required in order to efficiently solve the problem. More precisely, an efficient computation procedure for computing (9) is required. This is a nontrivial task for many application and requires problem specific research.
Footnotes
 1.
All simulations were carried out with Intel octa core i73770 CPU 3.40GHz and 16 GB RAM using MATLAB.
Notes
Acknowledgements
We gratefully acknowledge the support of the Swedish Research Council under Grant No. 62120134640.
References
 1.Abramowitz, M., Stegun, I.: Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables, vol. 55. Courier Corporation, North Chelmsford (1964)zbMATHGoogle Scholar
 2.Bai, Z., Demmel, J., Dongarra, J., Ruhe, A., van der Vorst, H.: Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, vol. 11. Siam, New Delhi (2000)CrossRefzbMATHGoogle Scholar
 3.Bai, Z., Su, Y.: SOAR: A secondorder Arnoldi method for the solution of the quadratic eigenvalue problem. SIAM J. Matrix Anal. Appl. 26(3), 640–659 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
 4.Beckermann, B.: The condition number of real Vandermonde, Krylov and positive definite Hankel matrices. Numer. Math. 85(4), 553–577 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
 5.Betcke, M.M., Voss, H.: Restarting projection methods for rational eigenproblems arising in fluidsolid vibrations. Math. Model. Anal. 13(2), 171–182 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
 6.Betcke, M.M., Voss, H.: Restarting iterative projection methods for Hermitian nonlinear eigenvalue problems with minmax property. Numer. Math. 135(2), 397–430 (2017)Google Scholar
 7.Betcke, T., Higham, N.J., Mehrmann, V., Schröder, C., Tisseur, F.: NLEVP: a collection of nonlinear eigenvalue problems. In: Technical Report, Manchester Institute for Mathematical Sciences (2011)Google Scholar
 8.Betcke, T., Voss, H.: A Jacobi–Davidsontype projection method for nonlinear eigenvalue problems. Future Gener. Comput. Syst. 20(3), 363–372 (2004)CrossRefGoogle Scholar
 9.Effenberger, C.: Robust solution methods for nonlinear eigenvalue problems. Ph.D. thesis, École polytechnique fédérale de Lausanne (2013)Google Scholar
 10.Golub, G.H., Van Loan, C., Charles, F.: Matrix Computations, vol. 3. JHU Press, Baltimore (2012)zbMATHGoogle Scholar
 11.Güttel, S., Van Beeumen, R., Meerbergen, K., Michiels, W.: NLEIGS: a class of fully rational Krylov methods for nonlinear eigenvalue problems. SIAM J. Sci. Comput. 36(6), A2842–A2864 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
 12.Jarlebring, E., Meerbergen, K., Michiels, W.: Computing a partial Schur factorization of nonlinear eigenvalue problems using the infinite Arnoldi method. SIAM J. Matrix Anal. Appl. 35(2), 411–436 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
 13.Jarlebring, E., Mele, G., Runborg, O.: The waveguide eigenvalue problem and the tensor infinite Arnoldi method. SIAM J. Sci. Comput. 39(3), A1062–A1088 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
 14.Jarlebring, E., Michiels, W., Meerbergen, K.: A linear eigenvalue algorithm for the nonlinear eigenvalue problem. Numer. Math. 122(1), 169–195 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
 15.Jarlebring, E., Poloni, F.: Iterative methods for the delay Lyapunov equation with TSylvester preconditioning. In: Technical Report (2015). ArXiv:1507.02100
 16.Kressner, D.: A block Newton method for nonlinear eigenvalue problems. Numer. Math. 114(2), 355–372 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
 17.Kressner, D., Roman, J.E.: Memoryefficient Arnoldi algorithms for linearizations of matrix polynomials in Chebyshev basis. Numer. Linear Algeb. Appl. 21(4), 569–588 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
 18.Lancaster, P., Psarrakos, P.: On the pseudospectra of matrix polynomials. SIAM J. Matrix Anal. Appl. 27(1), 115–129 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
 19.Lehoucq, R.B.: Analysis and implementation of an implicitly restarted Arnoldi iteration. Ph.D. thesis, Rice University (1995)Google Scholar
 20.Lehoucq, R.B., Sorensen, D.C.: Deflation techniques for an implicitly restarted Arnoldi iteration. SIAM J. Matrix Anal. Appl. 17(4), 789–821 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
 21.Lu, D., Su, Y., Bai, Z.: Stability analysis of the twolevel orthogonal Arnoldi procedure. SIAM J. Matrix Anal. Appl. 37(1), 195–214 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
 22.Mackey, D.S., Mackey, N., Mehl, C., Mehrmann, V.: Structured polynomial eigenvalue problems: good vibrations from good linearizations. SIAM J. Matrix Anal. Appl. 28(4), 1029–1051 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
 23.Mackey, D.S., Mackey, N., Tisseur, F.: Polynomial eigenvalue problems: Theory, computation, and structure. In: Numerical Algebra, Matrix Theory, DifferentialAlgebraic Equations and Control Theory, pp. 319–348. Springer (2015)Google Scholar
 24.Meerbergen, K.: Locking and restarting quadratic eigenvalue solvers. SIAM J. Sci. Comput. 22(5), 1814–1839 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
 25.Meerbergen, K.: The quadratic Arnoldi method for the solution of the quadratic eigenvalue problem. SIAM J. Matrix Anal. Appl. 30(4), 1463–1482 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
 26.Mehrmann, V., Voss, H.: Nonlinear eigenvalue problems: a challenge for modern eigenvalue methods. GAMM Mitt. 27(2), 121–152 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
 27.Morgan, R.: On restarting the Arnoldi method for large nonsymmetric eigenvalue problems. Math. Comput. 65(215), 1213–1230 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
 28.Neumaier, A.: Residual inverse iteration for the nonlinear eigenvalue problem. SIAM J. Numer. Anal. 22(5), 914–923 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
 29.Stewart, G.W.: A Krylov–Schur algorithm for large eigenproblems. SIAM J. Matrix Anal. Appl. 23(3), 601–614 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
 30.Su, Y., Bai, Z.: Solving rational eigenvalue problems via linearization. SIAM J. Matrix Anal. Appl. 32(1), 201–216 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
 31.Szyld, D., Vecharynski, E., Xue, F.: Preconditioned eigensolvers for largescale nonlinear Hermitian eigenproblems with variational characterizations. II. Interior eigenvalues. SIAM J. Sci. Comput. 37(6), A2969–A2997 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
 32.Szyld, D., Xue, F.: Preconditioned eigensolvers for largescale nonlinear Hermitian eigenproblems with variational characterizations. I. Extreme eigenvalues. Math. Comput. 85(302), 2887–2918 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
 33.Tisseur, F., Meerbergen, K.: The quadratic eigenvalue problem. SIAM Rev. 2, 235–286 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
 34.Van Beeumen, R., Jarlebring, E., Michiels, W.: A rankexploiting infinite Arnoldi algorithm for nonlinear eigenvalue problems. Numer. Linear Algebra Appl. 23(4), 607–628 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
 35.Van Beeumen, R., Meerbergen, K., Michiels, W.: Compact rational Krylov methods for nonlinear eigenvalue problems. SIAM J. Matrix Anal. Appl. 36(2), 820–838 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
 36.Voss, H.: A maxmin principle for nonlinear eigenvalue problems with application to a rational spectral problem in fluidsolid vibration. Appl. Math. 48(6), 607–622 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
 37.Voss, H.: An Arnoldi method for nonlinear eigenvalue problems. BIT 44(2), 387–401 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
 38.Voss, H.: Nonlinear eigenvalue problems. In: L. Hogben (ed.) Handbook of Linear Algebra, Second Edition, no. 164 in Discrete Mathematics and Its Applications. Chapman and Hall/CRC (2013)Google Scholar
 39.Zhang, Y., Su, Y.: A memoryefficient model order reduction for timedelay systems. BIT 53(4), 1047–1073 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.