BIT Numerical Mathematics

, Volume 19, Issue 2, pp 145–163 | Cite as

Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations

  • Åke Björck
  • Tommy Elfving


Iterative methods are developed for computing the Moore-Penrose pseudoinverse solution of a linear systemAx=b, whereA is anm ×n sparse matrix. The methods do not require the explicit formation ofATA orAAT and therefore are advantageous to use when these matrices are much less sparse thanA itself. The methods are based on solving the two related systems (i)x=ATy,AATy=b, and (ii)ATAx=ATb. First it is shown how theSOR-andSSOR-methods for these two systems can be implemented efficiently. Further, the acceleration of theSSOR-method by Chebyshev semi-iteration and the conjugate gradient method is discussed. In particular it is shown that theSSOR-cg method for (i) and (ii) can be implemented in such a way that each step requires only two sweeps through successive rows and columns ofA respectively. In the general rank deficient and inconsistent case it is shown how the pseudoinverse solution can be computed by a two step procedure. Some possible applications are mentioned and numerical results are given for some problems from picture reconstruction.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. S. Anderssen and G. H. Golub,Richardson's non-stationary matrix iterative procedure, Report STAN-CS-72-304, Stanford (1972).Google Scholar
  2. 2.
    V. Ashkenazi,Geodetic normal equations, 57–74, inLarge Sparse Sets of Linear Equations, ed. J. K. Reid, Academic Press, New York (1971).Google Scholar
  3. 3.
    O. Axelsson,On preconditioning and convergence accelerations in sparse matrix problems, CERN 74–10, Geneva (1974).Google Scholar
  4. 4.
    O. Axelsson,Solution of linear systems of equations: iterative methods, inSparse Matrix Techniques, ed. V. A. Barker, Lectures Notes in Mathematics 572, Springer-Verlag (1977).Google Scholar
  5. 5.
    Å. Björck,Methods for sparse linear least squares problems, inSparse Matrix Computations, eds. J. R. Bunch and D. J. Rose, Academic Press, New York (1976).Google Scholar
  6. 6.
    Y. T. Chen,Iterative methods for linear least squares problems, Ph. D. dissertation, Dep. Comput. Sci., Waterloo, Report CS-75-04 (1975).Google Scholar
  7. 7.
    R. J. Clasen,A note on the use of the conjugate gradient method in the solution of a large system of sparse equations, Computer J. 20 (1977), 185–186.Google Scholar
  8. 8.
    P. Concus, G. H. Colub and D. O'Leary,A generalized conjugate gradient method for the numerical solution of elliptic partial differential equations, Report STAN-CS-75-535, Stanford (1975).Google Scholar
  9. 9.
    J. Dyer,Acceleration of the convergence of the Kaczmarz method and iterated homogeneous transformation, Ph.D. thesis, UCLA, Los Angeles (1965).Google Scholar
  10. 10.
    T. Elfving,On the conjugate gradient method for solving linear least squares problems, Report LiTH-MAT-R-1978-3, Linköping (1978).Google Scholar
  11. 11.
    T. Elfving,Group-iterative methods for consistent and inconsistent linear equations, Report LiTH-MAT-R-1977-11, Revised 1978-02-10, Linköping (1978).Google Scholar
  12. 12.
    S. Erlander,Entropy in linear programs — an approach to planning, Report LiTH-MAT-R-1977-3, Linköping (1977).Google Scholar
  13. 13.
    V. Friedrich, Zur iterativen Behandlung unterbestimmter und nichtkorrekter linearen Aufgaben, Beiträge zur Numerischen Mathematik, 3 11–20, Oldenburg Verlag, München - Wien (1975).Google Scholar
  14. 14.
    A. de la Garza,An iterative method for solving systems of linear equations, Union Carbide, Oak Ridge, Report K-731, Tennessee (1951).Google Scholar
  15. 15.
    T. Ginsburg,The conjugate gradient method, inHandbook for Automatic Computation Vol. II,Linear Algebra, eds. J. H. Wilkinson and C. Reinsch, Springer-Verlag (1971).Google Scholar
  16. 16.
    G. T. Herman, A. Lent and S. W. Rowland,ART: Mathematics and Applications, J. Theor, Biol. 42 (1973), 1–32.Google Scholar
  17. 17.
    M. R. Hestenes,Pseudoinverses and conjugate gradients, Comm. of the ACM 18 (1975), 40–43.Google Scholar
  18. 18.
    M. R. Hestenes and E. Stiefel,Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, Sect. B 49 (1952), 409–436.Google Scholar
  19. 19.
    A. S. Householder and F. L. Bauer,On certain iterative methods for solving linear systems, Numer. Math. 2 (1960), 55–59.Google Scholar
  20. 20.
    S. Kaczmarz,Angenäherte Auflösung von Systemen linearer Gleichungen, Bull. Internat. Acad. Polon. Sciences et Lettres (1937), 355–357.Google Scholar
  21. 21.
    W. J. Kammerer and M. Z. Nashed,On the convergence of the conjugate gradient method for singular linear operator equations, SIAM J. Numer. Anal. 9 (1972), 165–181.Google Scholar
  22. 22.
    H. B. Keller,On the solution of singular and semidefinite linear systems by iteration, J. SIAM Numer. Anal. 2 (1965), 281–290.Google Scholar
  23. 23.
    C. Lanozos,Solution of linear equations by minimized iteration, Journal of Research of the National Bureau of Standards 49 Sect. B, (1952), 33–53.Google Scholar
  24. 24.
    P. Läuchli, Iterative Lösung und Fehlerabschätzung in der Ausgleichsrechnung, Z. für angew. Math. und Physik 10 (1959), 245–280.Google Scholar
  25. 25.
    C. C. Paige and M. A. Saunders,Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal. 12 (1975), 617–629.Google Scholar
  26. 26.
    W. Peters, Lösung linearer Gleichungssysteme durch Projektion auf Schnitträume von Hyperebenen und Berechnung einer verallgemeinerten Inversen, Beiträge zur Numerischen Mathematik 5 (1976), 129–146.Google Scholar
  27. 27.
    R. J. Plemmons,Stationary iterative methods for linear systems with non-Hermitian singular matrices, Report from Dept. of Comp. Science, The University of Tennessee, Knoxville, Tennessee.Google Scholar
  28. 28.
    J. K. Reid,On the method of conjugate gradients for the solution of large sparse systems of linear equations, inLarge Sparse Sets of Linear Equations, ed. J. K. Reid, Academic Press, New York (1971).Google Scholar
  29. 29.
    H. Rutishauser,Theory of gradient methods, inRefined Iterative Methods for Computation of the Solution and the Eigenvalues of Self-Adjoint Boundary Value Problems, M. Engeli et al., Birkhäuser, Basel (1959).Google Scholar
  30. 30.
    H. R. Schwarz, Die Methode der konjugierten Gradienten in der Ausgleichsrechnung, Zeitschrift für Vermessungswesen 95 (1970), 130–140.Google Scholar
  31. 31.
    E. Stiefel, Ausgleichung ohne Aufstellung der Gausschen Normalgleichungen, Wiss. Z. Technische Hochschule Dresden 2 (1952/3), 441–442.Google Scholar
  32. 32.
    G. W. Stewart,Introduction to Matrix Computation, Academic Press, New York (1973).Google Scholar
  33. 33.
    K. Tanabe,Projection method for solving a singular system of linear equations and its application, Numer. Math.17 (1971), 203–217.Google Scholar
  34. 34.
    A. van der Sluis,Condition numbers and equilibration of matrices, Numer. Math. 14 (1969), 14–23.Google Scholar
  35. 35.
    T. M. Whitney and R. K. Meany,Two algorithms related to the method of steepest descent, SIAM J. Numer. Anal. 4 (1967), 109–118.Google Scholar
  36. 36.
    H. Wozniakowski,Numerical stability of the Chebyshev method for the solution of large linear systems, Numer. Math. (1977), 191–209.Google Scholar
  37. 37.
    D. M. Young,Iterative solution of large linear systems, Academic Press, New York (1971).Google Scholar

Copyright information

© BIT Foundations 1979

Authors and Affiliations

  • Åke Björck
    • 1
  • Tommy Elfving
    • 1
  1. 1.Department of MathematicsLinköping UniversityLinköpingSweden

Personalised recommendations