Numerical Algorithms

, Volume 29, Issue 1–3, pp 229–247 | Cite as

On the Vector ε-Algorithm for Solving Linear Systems of Equations

  • A. Salam
  • P.R. Graves-Morris


The four vector extrapolation methods, minimal polynomial extrapolation, reduced rank extrapolation, modified minimal polynomial extrapolation and the topological epsilon algorithm, when applied to linearly generated vector sequences are Krylov subspace methods and it is known that they are equivalent to some well-known conjugate gradient type methods. However, the vector ε-algorithm is an extrapolation method, older than the four extrapolation methods above, and no similar results are known for it. In this paper, a determinantal formula for the vector ε-algorithm is given. Then it is shown that, when applied to a linearly generated vector sequence, the algorithm is also a Krylov subspace method and for a class of matrices the method is equivalent to a preconditioned Lanczos method. A new determinantal formula for the CGS is given, and an algebraic comparison between the vector ε-algorithm for linear systems and CGS is also given.

extrapolation vector Padé approximation projection methods Krylov subspace methods Lanczos methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    W.E. Arnoldi, The principle of minimized iterations in the solution of the matrix eigenvalue problem, Quart. Appl. Math. 9 (1951) 17–29.Google Scholar
  2. [2]
    O. Axelsson, Conjugate gradient type methods for unsymmetric and inconsistent systems of linear equations, Linear Algebra Appl. 29 (1980) 1–16.Google Scholar
  3. [3]
    C. Brezinski, Some results in the theory of the vector e-algorithm, Linear Algebra Appl. 8 (1974) 77–86.Google Scholar
  4. [4]
    C. Brezinski, Généralisations de la transformation de Shanks, de la table de Padé et de l' e-algorithme, Calcolo 12 (1975) 317–360.Google Scholar
  5. [5]
    C. Brezinski, Padé-Type Approximation and General Orthogonal Polynomials (Birkhäuser, Basel, 1980).Google Scholar
  6. [6]
    C. Brezinski and M. Redivo-Zaglia, Treatment of near-breakdown in the CGS algorithms, Numer. Algorithms 7 (1994) 33–73.Google Scholar
  7. [7]
    C. Brezinski and M. Redivo-Zaglia, Look-ahead in BiCGSTAB and other product-type methods for linear systems, BIT 35 (1995) 169–201.Google Scholar
  8. [8]
    C. Brezinski and M. Redivo-Zaglia, Extrapolation Methods. Theory and Practice (North-Holland, Amsterdam, 1996).Google Scholar
  9. [9]
    C. Brezinski and H. Sadok, Avoiding breakdown in the CGS algorithm, Numer. Algorithms 1 (1991) 199–206.Google Scholar
  10. [10]
    C. Brezinski and H. Sadok, Lanczos-type algorithms for solving systems of linear equations, Appl. Numer. Math. 11 (1993) 443–473.Google Scholar
  11. [11]
    S. Cabay and L.W. Jackson, A polynomial extrapolation method for finding limits and antilimits of vector sequences, SIAM J. Numer. Anal. 13 (1976) 734–752.Google Scholar
  12. [12]
    R.P. Eddy, Extrapolating to the limit of a vector sequence, in: Information Linkage between Applied Mathematics and Industry, ed. P.C.C. Wang (Academic Press, New York, 1979) pp. 387–396.Google Scholar
  13. [13]
    S.C. Eisenstat, H.C. Elman and M.H. Schultz, Variational iterative methods for nonsymmetric systems of linear equations, SIAM J. Numer. Anal. 20 (1983) 345–357.Google Scholar
  14. [14]
    W. Gander, G.H. Golub and D. Gruntz, Solving linear equations by extrapolation, in: Supercomputing, Trondheim, 1989, Computer Systems Science, Vol. 62 (Springer, Berlin, 1989) pp. 279–293.Google Scholar
  15. [15]
    P.R. Graves-Morris, Vector valued rational interpolants I, Numer. Math. 42 (1983) 331–348.Google Scholar
  16. [16]
    P.R. Graves-Morris, Vector valued rational interpolants II, IMA J. Numer. Anal. 4 (1984) 209–224.Google Scholar
  17. [17]
    P.R. Graves-Morris, G.A. Baker Jr. and C.F. Woodcock, Cayley's theorem and its application in the theory of vector Padé approximants, J. Comput. Appl. Math. 66 (1996) 255–265.Google Scholar
  18. [18]
    P.R. Graves-Morris and C.D. Jenkins, Vector-valued, rational interpolants III, Constr. Approx. 2 (1986) 263–289.Google Scholar
  19. [19]
    C. Lanczos, Solution of systems of linear equations by minimized iteration, J. Res. N.B.S. 49 (1952) 33–53.Google Scholar
  20. [20]
    J.B. McLeod, A note on the e-algorithm, Computing 7 (1971) 17–24.Google Scholar
  21. [21]
    M. Mešina, Convergence acceleration for the iterative solution of the equations X= AX+ f, Comput. Methods Appl. Mech. Engrg. 10 (1977) 165–173.Google Scholar
  22. [22]
    J. Nuttall, Convergence of Padé approximants of meromorphic functions, J. Math. Anal. Appl. 31 (1970) 147–153.Google Scholar
  23. [23]
    D.E. Roberts, Clifford algebras and vector-valued rational forms I, Proc. Roy. Soc. London A 431 (1990) 285–300.Google Scholar
  24. [24]
    D.E. Roberts, Clifford algebras and vector-valued rational forms II, Numer. Algorithms 3 (1992) 371–381.Google Scholar
  25. [25]
    D.E. Roberts, On a q-d algorithm, Adv. Comput. Math. 8 (1998) 193–219.Google Scholar
  26. [26]
    Y. Saad, Krylov subspace methods for solving large unsymmetric linear systems, Math. Comp. 37 (1981) 105–126.Google Scholar
  27. [27]
    Y. Saad, Iterative Methods for Sparse Linear Systems (PWS, Boston, 1996).Google Scholar
  28. [28]
    A. Salam, Formal vector orthogonal polynomials, Adv. Comput. Math. 8 (1988) 267–289.Google Scholar
  29. [29]
    A. Salam, Vector Padé-type approximants and vector Padé approximants, J. Approx. Theory 97 (1999) 92–112.Google Scholar
  30. [30]
    A. Salam, On vector Hankel determinant, Linear Algebra Appl. 313 (2000) 127–139.Google Scholar
  31. [31]
    D. Shanks, Non-linear transformations of divergent and slowly convergent sequences, J. Math. Phys. 34 (1955) 1–42.Google Scholar
  32. [32]
    A. Sidi, Convergence and stability properties of minimal polynomial and reduced rank extrapolation algorithms, SIAM J. Numer. Anal. 23 (1986) 197–209.Google Scholar
  33. [33]
    A. Sidi, Extrapolation vs. projection methods for linear systems of equations, J. Comput. Appl. Math. 22 (1988) 71–88.Google Scholar
  34. [34]
    A. Sidi, W.F. Ford and D.A. Smith, Acceleration of convergence of vectors sequences, SIAM J. Numer. Anal. 23 (1986) 178–196.Google Scholar
  35. [35]
    D.A. Smith, W.F. Ford and A. Sidi, Extrapolation methods for vector sequences, SIAM J. Numer Anal. 23 (1986) 178–196.Google Scholar
  36. [36]
    P. Sonneveld, CGS, A fast Lanczos-type solver for nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 10 (1989) 35–52.Google Scholar
  37. [37]
    H.A. Van Der Vorst, BiCGSTAB: A fast and smoothly converging variant of the Bi-CG for the solution of nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 13 (1992) 631–644.Google Scholar
  38. [38]
    P. Wynn, On a device for computing the e m (S n )transformation, Math. Tables Autom. Comp. 10 (1956) 91–96.Google Scholar
  39. [39]
    P. Wynn, Acceleration techniques for iterated vector and matrix problems, Math. Comput. 16 (1962) 301–322.Google Scholar
  40. [40]
    P. Wynn, Vector continued fractions, Linear Algebra Appl. 1 (1968) 357–395.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • A. Salam
    • 1
  • P.R. Graves-Morris
    • 2
  1. 1.Laboratoire de Mathématiques Pures et Appliquées J. LiouvilleUniversité du Littoral, Centre Universitaire de la Mi-Voix, Bâtiment PoincaréCalais CedexFrance
  2. 2.Department of ComputingUniversity of BradfordWest YorkshireEngland

Personalised recommendations