Skip to main content
SpringerLink
Log in
Book cover

Parallel Numerical Algorithms pp 91–118Cite as

Linear System Solvers: Sparse Iterative Methods

  • Chapter

Part of the book series: ICASE/LaRC Interdisciplinary Series in Science and Engineering ((ICAS,volume 4))

Abstract

In this chapter we present an overview of a number of related iterative methods for the solution of linear systems of equations. These methods are of Krylov projection type and include such popular methods as Conjugate Gradients, Bi-Conjugate Gradients, LSQR, and GMRES. We will sketch how these methods can be derived from simple basic iteration formulas, and how they are interrelated.

Iterative schemes are usually considered as alternatives for the solution of linear sparse systems, like those arising in, e.g., finite element or finite difference approximation of (systems of) partial differential equations. The structure of the operators plays no explicit role in any of these schemes, and the operator may be given even as a rule or a subroutine.

Although these methods seem to be almost trivially parallelizable at first glance, parallelism is sometimes a point of concern because of the inner products involved. We will consider this point in some detail.

Iterative methods are usually applied in combination with so-called preconditioning operators in order to improve convergence properties. This aspect receives more attention in our other chapter in this volume.

The work of this author was partially supported by the National Science Foundation under contract ASC 92-01266, the Army Research Office under contracts DAAL03-91-G-0150 and DAAL03-91-C-0047 (Univ. Tenn. subcontract ORA4466.04 Amendment 1), and the Office of Naval Research under contract ONR-N00014-92-J-1890.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Bank, R. and Chan, T. F., 1993. “An analysis of the composite step bi-conjugate gradient method,” Numer. Math. 66, pp. 295–319.

    Article  MathSciNet  MATH  Google Scholar 

  • Barrett, R., Berry, M., Chan, T., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., and Van der Vorst, H., 1994. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM, Philadelphia.

    Book  Google Scholar 

  • Brezinski, C. and Redivo-Zaglia, M., 1994. “Treatment of near breakdown in the CGS algorithm,” Numerical Algorithms 7, pp. 33–73.

    Article  MathSciNet  MATH  Google Scholar 

  • Chan, T. F. and Szeto, T., 1994a. “A composite step conjugate gradient squared algorithm for solving nonsymmetric linear systems,” Numerical Algorithms 7, pp. 16–32.

    Article  MathSciNet  Google Scholar 

  • Chan, T. F. and Szeto, T., 1994b. “Composite step product methods for solving nonsymmetric linear systems,” Technical Report CAM-94-18, UCLA, Department of Mathematics, Los Angeles.

    Google Scholar 

  • Chronopoulos, A. T. and Kim, S. K., 1990. “s-Step Orthomin and GM RES implemented on parallel computers,” Technical Report 90/43R, UMSI, Minneapolis.

    Google Scholar 

  • De Sturler, E., 1991. “A parallel restructured version of GMRES(m), ” Technical Report 91-85, Delft University of Technology, Delft.

    Google Scholar 

  • De Sturler, E. and Fokkema, D. R., 1993. “Nested Krylov methods and preserving the orthogonality,” Preprint 796, Utrecht University, Department of Mathematics, Utrecht.

    Google Scholar 

  • Demmel, J., Heath, M., and Van der Vorst, H., 1993. “Parallel linear algebra,” Acta Numerica 2, Cambridge University Press, Cambridge.

    Google Scholar 

  • Dongarra, J. J., 1990. “Performance of various computers using standard linear equations software in a Fortran environment,” Technical Report CS-89-85, University of Tennessee, Knoxville.

    Google Scholar 

  • Dongarra, J. J., Duff, I. S., Sorensen, D. C., and Van der Vorst, H. A., 1991. Solving Linear Systems on Vector and Shared Memory Computers, SIAM, Philadelphia.

    Google Scholar 

  • Dongarra, J. J. and Van der Vorst, H. A., 1992. “Performance of various computers using standard sparse linear equations solving techniques,” Supercomputer 9, pp. 17–29.

    Google Scholar 

  • Fletcher, R., 1976. “Conjugate gradient methods for indefinite systems,” Lecture Notes in Mathematics 506, Springer Verlag, Berlin, pp. 73–89.

    Google Scholar 

  • Freund, R. W., Golub, G. H., and Nachtigal, N. M., 1992. “Iterative solution of linear systems,” Acta Numerica 1, Cambridge University Press, Cambridge, pp. 57–100.

    Google Scholar 

  • Freund, R. W., 1993. “A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems,” SIAM J. Sci. Comp. 14, pp. 470–482.

    Article  MathSciNet  MATH  Google Scholar 

  • Golub, G. H. and Van Loan, C. F., 1989. Matrix Computations, The Johns Hopkins University Press, Baltimore.

    MATH  Google Scholar 

  • Gutknecht, M. H., 1993. “Variants of BICGSTAB for matrices with complex spectrum,” SIAM J. Sci. Comp. 14, pp. 1020–1033.

    Article  MathSciNet  MATH  Google Scholar 

  • Hestenes, M. R. and Stiefel, E., 1954. “Methods of conjugate gradients for solving linear systems,” J. Res. Natl. Bur. Stand. 49, pp. 409–436.

    Article  MathSciNet  Google Scholar 

  • Lanczos, C., 1952. “Solution of systems of linear equations by minimized iterations,” J. Res. Natl. Bur. Stand 49, pp. 33–53.

    Article  MathSciNet  Google Scholar 

  • Ortega, J. M., 1988. Introduction to Parallel and Vector Solution of Linear Systems, Plenum Press, New York.

    MATH  Google Scholar 

  • Paige, C. C. and Saunders, M. A., 1975. “Solution of sparse indefinite systems of linear equations,” SIAM J. Numer. Anal. 12, pp. 617–629.

    Article  MathSciNet  MATH  Google Scholar 

  • Paige, C. C. and Saunders, M. A., 1982. “LSQR: An algorithm for sparse linear equations and sparse least squares,” ACM Trans. Math. Soft. 8, pp. 43–71.

    Article  MathSciNet  MATH  Google Scholar 

  • Parlett, B. N., Taylor, D. R., and Liu, Z. A., 1985. “A look-ahead Lanczos algorithm for unsymmetric matrices,” Math. Comp. 44, pp. 105–124.

    MathSciNet  MATH  Google Scholar 

  • Pommerell, C., 1992. “Solution of large unsymmetric systems of linear equations,” Ph.D. thesis, Swiss Federal Institute of Technology, Zürich.

    Google Scholar 

  • Saad, Y., 1993. “A flexible inner-outer preconditioned GMRES algorithm,” SIAM J. Sci. Comp. 14, pp. 461–469.

    Article  MathSciNet  MATH  Google Scholar 

  • Saad, Y. and Schultz, M. H., 1986. “GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems,” SIAM J. Sci. Stat. Comp. 7, pp. 856–869.

    Article  MathSciNet  MATH  Google Scholar 

  • Simon, H. D., 1989. “Direct sparse matrix methods,” in Modern Numerical Algorithms for Supercomputers, Almond, J. C. and Young, D. M., eds., University of Texas at Austin, Center for High Performance Computing, pp. 325–444.

    Google Scholar 

  • Sleijpen, G. L. G., Van der Vorst, H. A., and Fokkema, D. R., 1994. “Bi-CGSTAB(l) and other hybrid bi-cg methods,” Numerical Algorithms 7, pp. 75–109.

    Article  MathSciNet  MATH  Google Scholar 

  • Sleijpen, G. L. G. and Fokkema, D. R., 1993. “BICGSTAB(l) for linear equations involving unsymmetric matrices with complex spectrum,” ETNA 1, pp. 11–32.

    MathSciNet  MATH  Google Scholar 

  • Sonneveld, P. 1989. “CGS: A fast Lanczos-type solver for non-symmetric linear systems,” SIAM J. Sci. Stat. Comp. 10, pp. 36–52.

    Article  MathSciNet  MATH  Google Scholar 

  • Van der Vorst, H. A., 1992. “Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems,” SIAM J. Sci. Stat. Comp. 13, pp. 631–644.

    Article  MATH  Google Scholar 

  • Van der Vorst, H. A. and Vuik, C., 1994. “GMRESR: A family of nested GMRES methods,” Num. Lin. Alg. Applies. 1, pp. 369–386.

    Article  MATH  Google Scholar 

  • Varga, R. S., 1962. Matrix Iterative Analysis, Prentice Hall, Engle-wood Cliffs.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics Institute, University of Utrecht, Utrecht, Netherlands

    Henk A. Van der Vorst

  2. Department of Mathematics, University of California at Los Angeles, Los Angeles, CA, 90024-1555, USA

    Tony F. Chan

Authors
  1. Henk A. Van der Vorst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tony F. Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Old Dominion University, Norfolk, Virginia, USA

    David E. Keyes

  2. University of Minnesota, Minneapolis, Minnesota, USA

    Ahmed Sameh

  3. ICASE, Hampton, Virginia, USA

    David E. Keyes  & V. Venkatakrishnan  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Van der Vorst, H.A., Chan, T.F. (1997). Linear System Solvers: Sparse Iterative Methods. In: Keyes, D.E., Sameh, A., Venkatakrishnan, V. (eds) Parallel Numerical Algorithms. ICASE/LaRC Interdisciplinary Series in Science and Engineering, vol 4. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-5412-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-94-011-5412-3_4

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-010-6277-0

  • Online ISBN: 978-94-011-5412-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation