Improvement of preconditioned bi-Lanczos-type algorithms with residual norm minimization for the stable solution of systems of linear equations

In this paper, improved algorithms are proposed for preconditioned bi-Lanczos-type methods with residual norm minimization for the stable solution of systems of linear equations. In particular, preconditioned algorithms pertaining to the bi-conjugate gradient stabilized method (BiCGStab) and the generalized product-type method based on the BiCG (GPBiCG) have been improved. These algorithms are more stable compared to conventional alternatives. Further, a stopping criterion changeover is proposed for use with these improved algorithms. This results in higher accuracy (lower true relative error) compared to the case where no changeover is done. Numerical results confirm the improvements with respect to the preconditioned BiCGStab, the preconditioned GPBiCG, and stopping criterion changeover. These improvements could potentially be applied to other preconditioned algorithms based on bi-Lanczos-type methods.

Preconditioned bi-Lanczos-type algorithms with residual norm minimization algorithms. In Sect. 5, we present the results of numerical experiments. Finally, conclusions are offered in Sect. 6.

Preliminaries
In this study, preconditioned algorithm and preconditioned system refer to solving algorithms described with some preconditioning operator M (or preconditioner/preconditioning matrix) and the system converted by the operator based on M , respectively. Here, M is not a specific preconditioner, but a general preconditioner.

Definition 1
The matrix, vector and scalar variable for a preconditioned system are denoted by tilde ( ̃ ). Then, for the system and solution we define the direction of the preconditioned system of linear equations as follows: -The two-sided preconditioned system: Equation Other vectors in the solution method are not preconditioned. The initial guess is given as x 0 , and x 0 = M r x 0 .
In terms of bi-Lanczos-type methods like the BiCG and the CGS, we assume the existence of a dual system A T x ♯ = b ♯ and we will refer to this as the "shadow system". It has the initial shadow residual vector (ISRV: r ♯ 0 (= b ♯ − A T x ♯ 0 ) assuming an initial guess). References [7,8] have reported the following theorem concerning the relation between the construction and setting of ISRV and preconditioned BiCG, and CGS methods. Theorem 1 (Itoh and Sugihara [7,8]) The direction of a preconditioned system for the BiCG method and CGS method is switched by the construction and setting of ISRV.
After preliminaries, Example 1 in Definition 8 is used to illuminate Theorem 1.

Definition 2 Notation for the direction of preconditioning conversion:
A w (= M −1 l AM −1 r ) : the matrix with two-sided preconditioning conversion. A r (= AM −1 ) : the matrix with right-preconditioning conversion.
The recurrences of the BiCG for the preconditioned system are Here, R k (̃) is the residual polynomial of degree k. Further, P k (̃) is the probing direction polynomial of degree k, that is, r BiCG k = R k (Ã)r 0 and p BiCG k = P k (Ã)r 0 . For example, in the left-PBiCG, (2.7) is shown as . We denote these polynomials with superscript "L" 1 , to indicate that these polynomials pertain to the left-preconditioned system [7].

Definition 7 (Notation for the preconditioned vector)
The symbol v + k means v + k ≡ M −1 v k . To distinguish between the direction of the preconditioned system, we for the left-preconditioned system ( L ). Here, Pol k (Ã)r 0 means the polynomial of degree k for ṽ k . On the other hand 2 , for the right-preconditioned system ( R ), we use v k to distinguish with Theorem 2 (Lanczos [10], Fletcher [3], Itoh and Sugihara [7]) The BiCG method for the preconditioned system satisfies the following conditions: Here, r ♯ i and p ♯ i are the vectors in the dual shadow system of r BiCG j and p BiCG j , respectively.
(2.9) ⟨r ♯ i ,r BiCG j ⟩ = 0 (i ≠ j), (biorthogonality), (2.10) ⟨p ♯ i ,Ãp BiCG j ⟩ = 0 (i ≠ j), (biconjugacy). 1 In a similar manner, we use "R" to indicate the right-preconditioned system and "W" to indicated the two-sided preconditioned system. On the other hand, we denote the direction of the preconditioning conversion in lower case. 2 For coding purposes, we do not need to specifically distinguish v + k and v k .

3
Definition 8 (Abbreviation of initial shadow residual vectors [7,8]) We assume that ⟨r ♯ 0 ,r 0 ⟩ ≠ 0 , e.g., r ♯ 0 =r 0 ≠ 0 , regarding the construction and setting of the ISRV for the preconditioned system. Here, r 0 means the initial residual vector for the preconditioned system of r 0 = b − Ax 0 . If r ♯ 0 = r ♯ 0 and r 0 = M −1 r 0 are given, then it reduces to r ♯ 0 = M −1 r 0 (ISRV1). This construction and setting of ISRV brings coLeft to the left-preconditioned system. If r ♯ 0 = M −T r ♯ 0 and r 0 = r 0 are given, then it reduces to . This brings coLeft to the right-precon- , this brings the right-preconditioned algorithm to the left-preconditioned system.

Example 1
It is sufficient to prove the following cases regarding the biorthogonality. The biconjugacy can be proven in a similar manner [7,8].

3
Preconditioned bi-Lanczos-type algorithms with residual norm minimization

Analyses for various preconditioned BiCGStab algorithms
First, we outline how to derive the BiCGStab method for the preconditioned system, and propose relevant theorems concerning the characteristics of the PBiCGStab. Next, we give outlines of six types of PBiCGStab algorithms. Finally, we summarize the characteristics of their mathematical structures.

Derivation of the BiCGStab method from PBiCG
The shadow residual vector in the PBiCGStab adopts s ♯ k = S k (Ã T )r ♯ 0 using the polynomial S k (̃) of degree k, the stabilized polynomial. s ♯ k is expanded as follows 4 : where, "lc" is an abbreviation of the leading coefficient; Deriving the bi-Lanczos-type method from the PBiCG, we apply (2.9) to the following inner product: Therefore, Next, the inner product (2.10) is transformed by Further, like deriving (3.2), the inner product of s ♯ k and Ãp k reduces the following with ⟨r ♯ i ,Ãp BiCG We state the polynomials of the residual vector and probing direction vector as Then, the following Proposition consists of (3.2) to (3.5):

Proposition 3
These PSTAB k and PSTAB k compose the BiCG part of the PBiCGStab, and they determine the direction of the preconditioned system [7]. This is the common characteristic which exists in all bi-Lanczos-type methods.
The residual vector and polynomial of the BiCGStab method for the preconditioned system, are expressed as follows: The stabilized polynomial of PBiCGStab is and contains no information regarding the direction of the preconditioned system, because PSTAB k and PSTAB k are not present. Next, we consider PBiCGStab from the viewpoint of congruency [8]. PSTAB k of the PBiCGStab has the following operation, The coefficient (3.8) defines the minimal residual (MR) operation of the polynomial of degree 1 that minimizes the norm of the residual vector in PBiCGStab, ||r STAB k+1 || [5,18].

Theorem 3 PSTAB
k has no congruency on the direction of the preconditioning conversion, but its direction of the preconditioned system depends on the PBiCG regardless of the direction of the preconditioning conversion.
Proof The three directions of the preconditioning conversion of (3.6), the two-sided (w), the left (l) and the right (r), are as follows, respectively:

3
Preconditioned bi-Lanczos-type algorithms with residual norm minimization These PSTAB k corresponding to the above are Therefore, there is no congruency on the direction of the preconditioning conversion. Next, we discuss the component of M −1 t k in (3.12), which is the two-sided preconditioning conversion of (3.8), to analyze the direction of the preconditioned system. The initial residual vector of t k is also r 0 = M −1 l r 0 . We assume the BiCG part of (3.7) to be the right preconditioned system (R), that is R R k+1 (Ã w )r 0 . Then Here, the superscript STAB of the polynomial S k is omitted. In contrast, we assume the BiCG part of (3.7) to be the left preconditioned system (L). Then This is as per Ã wt R k , Ã wt L k of (3.8). In any preconditioned system, they will reduce to w k of (3.12) superficially. Further, r k and l k also depend on the direction of the preconditioned system of the BiCG part. ◻
However, it is not problematic to display only k or PSTAB k by broadly interpreting it as " k with preconditioning conversion" when referring to k in other algorithms. Therefore, only the MR coefficient in the PBiCGStab, that is w k , l k or r k , has no congruence property in a narrow sense. Further, these directions of the preconditioned system depend on the BiCG part. ,

Theorem 4 The direction of a preconditioned system which depends on the BiCG part is determined by the coefficients of k and k in each PBiCGStab algorithm. These intrinsic operations are based on biorthogonality and biconjugacy. Further, the direction of PBiCGStab is switched by the construction and setting of the ISRV.
Proof In the polynomial expressions of vectors r STAB the polynomial with the direction of the preconditioned system is only the BiCG part. That is R k (Ã)r 0 and P k (Ã)r 0 . Here, the recurrences of R k (̃) , P k (̃) contain PSTAB k and PSTAB k composed by the operations of biorthogonality or biconjugacy; these are switched by the construction and setting of ISRV. ◻ The assertions in Theorem 4 were also noted in References [7,8] where the PCGS and the PBiCG were analyzed with the congruence property (see Theorem 1 in this paper). Therein, it was proclaimed that the direction of the preconditioned system is not determined only by the linear system, but also depends on the construction and setting of the ISRV, for the preconditioned algorithms or the bi-Lanczos-type system. In this paper, we have confirmed this for the preconditioned algorithms of interest without the congruence property.
We verify Theorem 4 by various PBiCGStab algorithms in the next subsection and provide numerical results in Sect. 5.

Various preconditioned BiCGStab algorithms
In this subsection, six types of PBiCGStab algorithms are outlined. The first two algorithms are the right-PBiCGStab and left-PBiCGStab. These are based on Definition 1 with an extended application of Theorem 1 to the PBiCGStab. Subsequently, new variants of algorithms are presented. The first two algorithms are coordinative to the left-PBiCGStab (coLeft), and the algorithm switches the right-PBiCGStab to the left-preconditioned system by the construction and setting of ISRV9. To analyze algorithms with different directions between the BiCG and MR parts, the final two algorithms are given as follows. One is an algorithm with the left-preconditioned system of the BiCG part and the right-preconditioning conversion of the MR part that is mathematically equivalent to ISRV9 (Case1) 5 . The other is an algorithm with the right-preconditioned system of the BiCG part and the left-preconditioning conversion of the MR part (Case2). These six algorithms have two kinds of preconditioning operations in their iterative components. The Right and Left algorithms are both derived trivially. Only the coLeft algorithm is presented on the new variant PBiCGStab in this manuscript. However, Case1 with the polynomial expression is presented at the end of this section as a reference. Descriptions of all algorithms are presented in Appendix A. In the subsection, we only discuss algorithms based on BiCGStab, and we do not display " ̃ " (tilde) or the superscript of "(P)STAB".

Right-and Left-PBiCGStab algorithm
It is straightforward to derive both the Right and Left algorithms by the preconditioning conversion based on Definition 1, and by expressly providing the construction and setting of the ISRV.

Algorithm 1 Right-PBiCGStab algorithm (Right):
The right-PBiCGStab is derived by Definition 1. This is the conventional PBiCGStab algorithm provided in References [1,12,18]. The construction and setting of the ISRV is r ♭ 0 = r 0 , and the polynomials of the right preconditioned system are formed [8]. The standard stopping criteria are

Algorithm 2 Left-PBiCGStab algorithm (Left):
The left-PBiCGStab is derived by Definition 1. The construction and setting of the ISRV is r ♯ 0 = r + 0 , and the polynomials of the left preconditioned system are formed [8]. The standard stopping criteria are

New variants of PBiCGStab algorithms
In this subsection, new variants of PBiCGStab algorithms are provided based on the foregoing Right and Left algorithms. For all algorithms, the standard stopping criteria are (3.15).
Algorithm 3 is derived as follows. Applying the same preconditioning conversion as Algorithm 2, splitting the residual vector r + k of Algorithm 2 into M −1 r k , and adding the recurrences for the residual vectors of t k = r k − k Ap + k and r k+1 = t k − k At + k . In this iterative part, both r k+1 and r + k+1 are calculated. The construction and setting of the ISRV is r ♯ 0 = M −1 r 0 . This algorithm is coordinative to the Left PBiCGStab, and this BiCG part is the left preconditioned system. The MR part is the left preconditioning conversion.

Algorithm 4 Right-PBiCGStab with ISRV9 (ISRV9) :
This algorithm is derived by replacing the ISRV of the Right (Algorithm 1) with . This BiCG part is the left preconditioned system and the MR part is the right preconditioning conversion.

Algorithm 5 BiCG-left and MR-right PBiCGStab algorithm (Case1):
This algorithm is mathematically equivalent to ISRV9 (Algorithm 4). That is, the BiCG part is the left preconditioned system and the MR part is the right preconditioning conversion.

Algorithm 6 BiCG-right and MR-left PBiCGStab algorithm (Case2):
This algorithm is composed of the opposite combination to Case1 (Algorithm 5). That is, the BiCG part is the right preconditioned system and the MR part is the left preconditioning conversion. Table 1 shows the characteristics of the PBiCGStab algorithms in this section.

Features of various preconditioned BiCGStab algorithms
There are differences between the Left (Algorithm 2) and the coLeft (Algorithm 3). The residual vector r + k of the Left is split into M −1 r k of the coLeft, and the stopping criteria are different (see, Definition 5). However, the true residual vector b − Ax k and the true error x k − x exact are mathematically equivalent between both algorithms. ISRV9 (Algorithm 4) is mathematically equivalent to Case1 (Algorithm 5). The Right is the same algorithm as ISRV9, except for the construction and setting of the ISRV. Therefore, we may confirm that the direction of a preconditioned system for the BiCGStab is switched by the construction and setting of the ISRV.
As a reference, Case1 (Algorithm 5) with the polynomial expression is shown; double underline is used to indicate the polynomials.

Algorithm 5' Polynomial description of the BiCG-left and MR-right PBiCG-Stab algorithm (Case 1) :
x 0 is an initial guess, End Do Table 1 Characteristics of the six types of PBiCGStab algorithms. In the column of k , k , "R" indicates the rightpreconditioned system and "L" indicates the left-preconditioned system. In the column of k , "r" indicates the right-direction of preconditioning conversion and "l" indicates the left-direction of preconditioning conversion

Analyses for various preconditioned GPBiCG algorithms
In this section, first we provide an outline for deriving the GPBiCG method for the preconditioned system, and propose some corollaries on the characteristics of the PGPBiCG. Next, we give outlines of the six types of PGPBiCG algorithms. These mathematical structure of the characteristics correspond to the six types of PBiCG-Stab in the former section.

Derivation of the GPBiCG method from PBiCG
Derivation of the GPBiCG for the preconditioned system proceeds in a similar manner as the derivation of the PBiCGStab in section 3. This is particularly the case with respect to the derivation procedure from (3.1) to (3.4); except that the superscripts of PSTAB

Corollary 1
The residual vector and polynomial of the GPBiCG method for the preconditioned system are expressed as follows: The stabilized polynomial of PGPBiCG is The coefficients of PGP k , PGP k compose the minimal residual (MR) operation of the polynomial of degree 2 that minimizes the norm of the residual vector in PGPBiCG, ||r GP k+1 || [4,5,19]. Therefore, the effect of the MR part in PGPBiCG is higher than in PBiCGStab.r

3
Preconditioned bi-Lanczos-type algorithms with residual norm minimization For projection onto the two-dimensional subspace spanned by ỹ k and Ãt k , we define the n × 2 matrix B ≡ ỹ k ,Ãt k . Hence, the optimal coefficients PGP k and PGP k for (4.1) are calculated as follows: After all, we obtain Then, the recurrences of (4.2) are transformed into the following: The following Corollary resonates with Theorem 3 of the PBiCGStab. The following are the coefficients of (4.3), (4.4) after preconditioning conversion to the right (r) and the left (l). .

Remark 2
The r k , r k of the right preconditioning conversion minimizes the norm of the original residual vector ‖r PGP k+1 ‖ without operating a preconditioner, as with Remark 1 for the PBiCGStab.
The following Corollary resonates with Theorem 4 of the PBiCGStab. We verify Corollary 3 by various PGPBiCG algorithms in the next subsection with numerical results provided in section 5.

Corollary 3
The direction of a preconditioned system which depends on the BiCG part is determined by the coefficients of k and k in each PGPBiCG algorithm. These intrinsic operations are based on biorthogonality and biconjugacy. Further, the direction of PGPBiCG is switched by the construction and setting of the ISRV.

Various preconditioned GPBiCG algorithms
In this subsection, the six types of PGPBiCG algorithms are shown and their mathematical structures are summarized in Sect. 4.2.3. These six algorithms have two kinds of preconditioning operations in their iterative part. Descriptions of all algorithms are presented in the Appendix B. We only discuss algorithms based on GPBiCG, and do not display " ̃ (tilde)" or the superscript of "(P)GP".

Right-and Left-PGPBiCG algorithm
The Right and Left algorithms are derived as follows.

Algorithm 7 Right-PGPBiCG algorithm (Right):
The "Right-PGPBiCG algorithm" based on Definition 1 has three kinds of preconditioning operations ( M −1 p k , M −1 t k , and M −1 z k ). We state z k ≡ M −1 z k , and use ȓ k (≡ M −1 r k ) , ȗ k (≡ M −1 u k ) . Further, we apply the necessary transformation. The construction and setting of the ISRV is r ♭ 0 = r 0 and the polynomials of the right preconditioned system are formed [8]. The standard stopping criteria are (3.15); ‖t k ‖∕‖b‖ ≤ , ‖r k+1 ‖∕‖b‖ ≤ .

New variants of PGPBiCG algorithms
In this subsection, new variants of PGPBiCG algorithms are provided. In the same manner as deriving new PBiCGStab variants, they are also based on the former Right (Algorithm 7) or Left (Algorithm 8) algorithms. For all algorithms, the standard stopping criteria are (3.15).

Algorithm 9 Coordinative to left-PGPBiCG algorithm (coLeft):
This algorithm is coordinative to the Left PGPBiCG and the BiCG part is the left preconditioned system. The MR part is the left preconditioning conversion. The construction and setting of the ISRV is r ♯ 0 = M −1 r 0 .

Algorithm 10 Right-PGPBiCG with ISRV9 (ISRV9):
This algorithm is derived by replacing the ISRV of the Right (Algorithm 7) with r ♭ 0 = M −T M −1 r 0 (ISRV9). The BiCG part is the left preconditioned system and the MR part is the right preconditioning conversion.

Algorithm 11 BiCG-left and MR-right PGPBiCG algorithm (Case 1):
This algorithm is mathematically equivalent to ISRV9 (Algorithm 10). That is, the BiCG part is the left preconditioned system and the MR part is the right preconditioning conversion.

Algorithm 12 BiCG-right and MR-left PGPBiCG algorithm (Case 2):
This algorithm is composed of the opposite combination to Case1 (Algorithm 11). That is, the BiCG part is the right preconditioned system and the MR part is the left preconditioning conversion. Table 2 shows the characteristics of the PGPBiCG algorithms in this section. Table 2 Characteristics of the six types of PGPBiCG algorithms. In the column of k , k , "R" indicates the rightpreconditioned system and "L" indicates the left-preconditioned system. In the column of k , k , " r " indicates the right-direction of preconditioning conversion and " l " indicates the leftdirection of preconditioning conversion Name (Alg. no.) ISRV k , k k , k Standard stopping criterion

Numerical experiments
In Sect. 5.1, we verify that the numerical values of k and k of PBiCG, PBiCGStab and PGPBiCG are equivalent for each direction of the preconditioned system (Proposition 3, Corollary 1). Further, we confirm the characteristics of the six types of algorithms (Right, Left, coLeft, ISRV9, Case1, and Case2), by the numerical values of k , k , k , and k . We also confirm that the direction of the preconditioned system which depends on the BiCG part is determined by the coefficients of k and k in each PBiCGStab, PGPBiCG algorithm, and the direction of PBiCGStab is switched by the construction and setting of the ISRV (Theorem 4, Corollary 3). In Sect. 5.2, we present numerical results associated with the six types of PBiCG-Stab and PGPBiCG in Sects. 3.2 and 4.2, and the numerical results of the three types of PBiCGStab and PGPBiCG with changing over of the stopping criterion [9].
The test problems were generated by building real nonsymmetric matrices corresponding to linear systems taken from the University of Florida Sparse Matrix Collection [2] and the Matrix Market [11]. The RHS vector b of (1.1) was generated in two ways: • RHSone] : Setting all elements of x exact as 1.0 and substituting this into (1.1). • RHS + nn] : Setting all elements of x exact as Mersenne Twister pseudo-random numbers and substituting this into (1.1). Here, we term this RHS "RHS + nn", because the random numbers are generated using a seed number (nn). Ex.) RHS + 10 means that b = Ax exact and x exact is generated by seed = 10.
The numerical experiments were executed on a DELL PowerEdge R515 (AMD Opteron 4133) running Cent OS (kernel 2.6.32) and Intel compiler 16.0.2. The solution algorithm was implemented using the sequential mode of the Lis numerical computation library 1.1. 2 [16] in double precision, with compiler options registered in the Lis "Makefile." In all tests, the initial solution was set to x 0 = 0 . ILU(0) was adopted as a preconditioning operation with PBiCG, PBiCGStab and PGPBiCG algorithms; here, the value "zero" means the fill-in level. We evaluated the algorithm's relative residual 2-norm (ARR). We adopted (3.15) as the stopping criterion in all cases except for the left-preconditioned algorithms where we used (3.16). We set = 10 −12 in all cases. In all tables, the true relative residual (TRR) log 10 2-norm ( ‖b − Ax‖∕‖b‖ ) and the true relative error (TRE) log 10 2-norm ( ‖x − x exact ‖∕‖x exact ‖ ) at convergence are stated. Here, x means the numerical solution. The maximum number of iterations (mx) was set to 1000, except for visco-plastic2 (mx = 2000) and young3c (mx = 2500) matrices solved by PBiCG-Stab, and young3c (mx = 5000) matrix solved by PGPBiCG.

Verification of PBiCG, PBiCGStab and PGPBiCG algorithms
The add32 matrix was used and the linear equations were generated using RHSone. We confirm Proposition 3 and Corollary 1. Figure 1 plots values of k and Fig. 2 plots values of k for PBiCG, PBiCGStab and PGPBiCG. In both figures, the upper displays the values of the right-preconditioned algorithms and the lower displays the values of the left-preconditioned algorithms.
We plotted the values of k and k for five algorithm types, except for coLeft 6 of PBiCGStab algorithms or PGPBiCG algorithms presented in Sects. 3.2 or 4.2; these are shown in Figs. 3, 4, 5, 6. From these figures 7 , we confirm the correspondence relations between the behavior of k and k and the direction of the preconditioned system summarized in Tables 1, 2. That is to say, behaviors of k and k are the same between Right and Case2 (right-preconditioned system algorithms), and the behaviors of k and k are the same among Left, ISRV9 and Case1 (leftpreconditioned system algorithms). On the other hand, behaviors of k and k are not the same between the right-preconditioned system algorithms and the left-preconditioned system algorithms. It is very important that the directions of the preconditioned system for ISRV9, Case1 and Case2 accord with the directions of each k and k in spite of the fact that the direction of the MR operators in these algorithms differ from the directions of each k and k ; see Theorem 3 and Corollary 2. Further, we have numerically confirmed Theorem 4 and Corollary 3, because the ISRV9 and Right algorithms are identical, except for the construction and setting of the ISRV.
We have also confirmed that both k values of the ISRV9 and Case1 algorithms of the PBiCGStab, the PGPBiCG, and k values of the PGPBiCG were the same in Figs. 7, 8 and 9; these algorithms were equivalent. In contrast, both values of the other algorithms of the above k and k differ from each other. We have again numerically confirmed Theorems 3 and 4, and Corollary 2, 3.
As reference information, other numerical results for add32 using the relative residual of each algorithm are listed in Appendix C.
In the next subsection, we will discuss various results by solving the six types of PBiCGStab and PGPBiCG.

PBiCGStab and PGPBiCG results
In this subsection, we compare and evaluate the performance of various PBiCG-Stab and PGPBiCG algorithms for solving the linear equations of RHSone. The results with the standard stopping criterion are shown in Tables 3, 4. Further, the results from using three types of algorithms (RtL, coLtL, C1tL) with stopping criterion changeover [9], are shown in Tables 5, 6. Here, "RtL" means the Right (Algorithm 1, 7) with stopping criterion (3.15) to the Left criterion (3.16) 8 by the changeover. In a similar way, "coLtL" means the coLeft to the Left criterion by the changeover, and "C1tL" means the Case1 to the Left criterion by the changeover.
In these tables, significant disadvantages of one algorithm over the other are emphasized by bold font. For example, poor accuracy over 10 −8 on the TRR or the    Fig. 6 Values of k for each PGPBiCG TRE, or too many iterations until convergence. Further, "Max" means reaching the maximum number of iterations with no convergence by stagnation or divergence. "BD" means breakdown at the operation of k or k . "NaN" means that "not a number" occurs at MR operations by zero division.
In Tables 3, 4, the results of Case2 are similar to Right in terms of breakdown and poor accuracy on the TRR and TRE. Here, Case2 is a kind of right-preconditioned system. We did not apply stopping criterion changeover to Case2, because the results using Case2 did not suggest an adequate solution. We applied stopping criterion changeover to coLeft and Case1 of the left-preconditioned system, and they are labeled "coLtL" and "C1tL", respectively. Because ISRV9 and Case1 are equivalent algorithms, we applied the criterion changeover only to Case1. Further, we applied the changeover to Right just for form, and this was labeled "RtL".
In Tables 5, 6, RtL is shown to yield improved accuracy for TRR and TRE compared to Right in many cases, but the result of viscoplastic2 in Table 6  Preconditioned bi-Lanczos-type algorithms with residual norm minimization Table 3 Numerical results using the relative residual of each algorithm. (PBiCGStab, RHSone); N is the problem size. The three numbers in each row for the column pertaining to each method are as follows: the leftmost number is log 10 of the TRR 2-norm, the number in parentheses is the required number of iterations before convergence, and the lower number is log  Table 4 Numerical results using the relative residual of each algorithm. (PGPBiCG, RHSone) The data in bold font shows a disadvantage, poor accuracy on TRR or TRE (over 10 −8 ), too many iterations required until convergence compared to the others, and BD, arc130 Table 6 Numerical results using the relative residual of each algorithm with stopping criterion changeover. (PGPBiCG, RHSone) The data in bold font shows a disadvantage, poor accuracy on TRR or TRE (over 10 −8 ), too many iterations required until convergence compared to the others, and BD, NaN, Max Matrix  Table 8 Numerical results using the relative residual of each algorithm with stopping criterion changeover for many RHSs. (viscoplastic1, PBiCGStab, mx = 1000) Preconditioned bi-Lanczos-type algorithms with residual norm minimization Table 9 Numerical results using the relative residual of each algorithm with stopping criterion changeover for many RHSs. (young3c, PBiCGStab, mx = 2500) suggests decreased accuracy for both. This is confirmed in Table 6 of PBiCGStab. Further, RtL did not lead to improved accuracy in Table 10. It was confirmed that RtL never solved equations that were not solved as BD or Max by Right (Tables 7,  11). Next, we consider the results pertaining to LTB in Definition 6 and list the problems for which Left was weak in Tables 3, 4. The LTB case is the result of poor accuracy on the TRE, in spite of convergence with a smaller or equal iteration number to coLeft, _ , { , } were concerned in Table  3 of PBiCGStab. Here, we extend the comparing target to ISRV9 and Case1, then young3c was also concerned 9 . Further, { , } were also concerned in Table 4 of PGPBiCG. We extend the comparing target to ISRV9 and Case1, then , _ were also concerned, and arc130 exhibited poor accuracy for the results of the six types of algorithms in Table 4. In terms of the other cases,   Tables 8, 9, 10, 11, 12.
Next, we consider results pertaining to GTB in Definition 6, and list the solving problems to work out by Left in Tables 3, 4. In Table 3 of PBiCGStab, arc130, olm{2000,5000}, raefsky3 were concerned, and Left exhibited superior accuracy compared to the other five types of algorithms 10 . In Table 4 of PGPBiCG, olm{2000,5000}, raefsky3 were concerned, and Left was the most accurate among the six types of algorithms. However, coLtL and C1tL converged roughly with fewer iterations and higher accuracy compared to Left in some cases as per Tables 5, 6. coLeft and coLtL are mathematically equivalent algorithms to Left, except for the stopping criterion being caused by splitting r + k of Left into M −1 r k . Comparing the results in Tables 5, 6, the case of the GTB and converging without NaN or Max, Left and coLtL exhibited mathematically identical accuracy; for RtL coLtL C1tL Fig. 11 Convergence histories for the TRR 2-norm (olm5000, RHSone) 10 The difference on the TRR and the TRE occurred for arc130 between the Left and coLeft in spite of the fact that the required numbers were both 5, because coLeft converged at the early check. example, arc130, olm{2000,5000}, raefsky3 in Table 5 (PBiCG-Stab), _ , , in Table 6 (PGPBiCG). This was also confirmed by the fact that the values after changing over coLeft to coLtL were the same as per Left in Fig. 10 (olm5000). We might roughly rank as "the coLeft ≤ the Left = the coLtL" on accuracy, except for the LTB cases. In the case of LTB, "the Left ≤ the coLeft = the coLtL", because there were many cases where coLeft achieved higher accuracy.
The Case1 and ISRV9 algorithms of the left-preconditioned system directly minimize the norm of the residual vector which is not operating M −1 in the MR part. These algorithms generated stable solutions without incurring breakdown, NaN or Max. However, a major gap between TRR and TRE was recognized at arc130, olm5000, raefsky3 in Table 3 (PBiCGStab) and at arc130,   Table 4 (PGPBiCG). This is because Case1 and ISRV9 adopted ‖r k+1 ‖∕‖b‖ as the stopping criterion, in spite of the relations between the TRE and the ARR of the algorithms of the left-preconditioned system, shown as (2.4) in Proposition 2. The above issue was resolved in the C1tL by the changeover. Further, improved accuracy was recognized. The issue of the major gap in the former case was also recognized in Left, however Left was not able to change over the stopping criterion. Figures 10,11,12,13,14,15 show the behavior of the ARR, TRR and TRE for x = x k+1 . In these figures, iterations of the TRR and the TRE continued irrespective of the second convergence of the ARR after changeover, to observe the converging status of the TRR and the TRE. From these results, we confirmed stagnation of  the TRR and TRE with high accuracy. We may expect high accuracy on the TRE without iteration stopping, but in practical terms, we need a mathematically based stopping criterion to avoid excessive iterative calculations, and the changeover was confirmed as an effective measure in this respect.
Finally, we summarize these numerical results. The changeover was very effective on the RtL, the coLtL and the C1tL. The results for Case1 and ISRV9, the algorithms composed by the BiCG part of the left-preconditioned system and the MR part of the right-preconditioning conversion, were stable in terms of accurately solving linear equations. C1tL further improved the accuracy of the TRE. From the above, C1tL is considered to be the best algorithm. We may think that "ISRV9 to the Left stopping criterion" is also promising because ISRV9 is equivalent to Case1. However, after the changeover, an early check is only available in the case of PBiCGStab, because the iteration part of ISRV9 is identical to Right.  In this paper, we have proposed improved PBiCGStab and PGPBiCG algorithms as preconditioned bi-Lanczos-type algorithms with residual norm minimization. It was confirmed that the direction of the preconditioned system which depends on the BiCG part is determined by the coefficients of k and k , in each PBiCGStab and PGPBiCG, similar to PBiCG or PCGS [7,8], in spite of the fact that both methods have MR operators of k and k with no congruency in terms of the direction of preconditioning convergence (Theorem 3, Corollary 2). We have numerically confirmed that the direction of the preconditioned system which depends on the BiCG part for both methods is switched by the construction and setting of the ISRV. That is, Theorem 1 for PBiCG and PCGS is extended to PBiCGStab and PGPBiCG RtL coLtL C1tL Fig. 15 Convergence histories for the TRE 2-norm (young3c, RHSone)

Table 10
Numerical results using the relative residual of each algorithm with stopping criterion changeover for many RHSs. (lop163, PGPBiCG, mx = 1000) Preconditioned bi-Lanczos-type algorithms with residual norm minimization Table 11 Numerical results using the relative residual of each algorithm with stopping criterion changeover for many RHSs. (olm5000, PGPBiCG, mx = 1000) The data in bold font shows a disadvantage, poor accuracy on TRR or TRE (over 10 −8 ), too many iterations required until convergence compared to the others, and BD, NaN, Max RHS ID

Table 12
Numerical results using the relative residual of each algorithm with stopping criterion changeover for many RHSs. (young3c, PGPBiCG, mx = 5000) The data in bold font shows a disadvantage, poor accuracy on TRR or TRE (over 10 −8 ), too many iterations required until convergence compared to the others, and BD, Numerical results for add32 using the relative residual of each algorithm. Tables 13,14,15 show numerical results for add32 as reference information 11 . 11 A difference in TRR and TRE occurred between the Left and the coLeft in spite of the fact that the required numbers were both 34 in Table 14. This is because coLeft converged at the early check.