Abstract
Iterative methods are developed for computing the Moore-Penrose pseudoinverse solution of a linear systemAx=b, whereA is anm ×n sparse matrix. The methods do not require the explicit formation ofA T A orAA T and therefore are advantageous to use when these matrices are much less sparse thanA itself. The methods are based on solving the two related systems (i)x=A T y,AA T y=b, and (ii)A T Ax=A T b. First it is shown how theSOR-andSSOR-methods for these two systems can be implemented efficiently. Further, the acceleration of theSSOR-method by Chebyshev semi-iteration and the conjugate gradient method is discussed. In particular it is shown that theSSOR-cg method for (i) and (ii) can be implemented in such a way that each step requires only two sweeps through successive rows and columns ofA respectively. In the general rank deficient and inconsistent case it is shown how the pseudoinverse solution can be computed by a two step procedure. Some possible applications are mentioned and numerical results are given for some problems from picture reconstruction.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
R. S. Anderssen and G. H. Golub,Richardson's non-stationary matrix iterative procedure, Report STAN-CS-72-304, Stanford (1972).
V. Ashkenazi,Geodetic normal equations, 57–74, inLarge Sparse Sets of Linear Equations, ed. J. K. Reid, Academic Press, New York (1971).
O. Axelsson,On preconditioning and convergence accelerations in sparse matrix problems, CERN 74–10, Geneva (1974).
O. Axelsson,Solution of linear systems of equations: iterative methods, inSparse Matrix Techniques, ed. V. A. Barker, Lectures Notes in Mathematics 572, Springer-Verlag (1977).
Å. Björck,Methods for sparse linear least squares problems, inSparse Matrix Computations, eds. J. R. Bunch and D. J. Rose, Academic Press, New York (1976).
Y. T. Chen,Iterative methods for linear least squares problems, Ph. D. dissertation, Dep. Comput. Sci., Waterloo, Report CS-75-04 (1975).
R. J. Clasen,A note on the use of the conjugate gradient method in the solution of a large system of sparse equations, Computer J. 20 (1977), 185–186.
P. Concus, G. H. Colub and D. O'Leary,A generalized conjugate gradient method for the numerical solution of elliptic partial differential equations, Report STAN-CS-75-535, Stanford (1975).
J. Dyer,Acceleration of the convergence of the Kaczmarz method and iterated homogeneous transformation, Ph.D. thesis, UCLA, Los Angeles (1965).
T. Elfving,On the conjugate gradient method for solving linear least squares problems, Report LiTH-MAT-R-1978-3, Linköping (1978).
T. Elfving,Group-iterative methods for consistent and inconsistent linear equations, Report LiTH-MAT-R-1977-11, Revised 1978-02-10, Linköping (1978).
S. Erlander,Entropy in linear programs — an approach to planning, Report LiTH-MAT-R-1977-3, Linköping (1977).
V. Friedrich, Zur iterativen Behandlung unterbestimmter und nichtkorrekter linearen Aufgaben, Beiträge zur Numerischen Mathematik, 3 11–20, Oldenburg Verlag, München - Wien (1975).
A. de la Garza,An iterative method for solving systems of linear equations, Union Carbide, Oak Ridge, Report K-731, Tennessee (1951).
T. Ginsburg,The conjugate gradient method, inHandbook for Automatic Computation Vol. II,Linear Algebra, eds. J. H. Wilkinson and C. Reinsch, Springer-Verlag (1971).
G. T. Herman, A. Lent and S. W. Rowland,ART: Mathematics and Applications, J. Theor, Biol. 42 (1973), 1–32.
M. R. Hestenes,Pseudoinverses and conjugate gradients, Comm. of the ACM 18 (1975), 40–43.
M. R. Hestenes and E. Stiefel,Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, Sect. B 49 (1952), 409–436.
A. S. Householder and F. L. Bauer,On certain iterative methods for solving linear systems, Numer. Math. 2 (1960), 55–59.
S. Kaczmarz,Angenäherte Auflösung von Systemen linearer Gleichungen, Bull. Internat. Acad. Polon. Sciences et Lettres (1937), 355–357.
W. J. Kammerer and M. Z. Nashed,On the convergence of the conjugate gradient method for singular linear operator equations, SIAM J. Numer. Anal. 9 (1972), 165–181.
H. B. Keller,On the solution of singular and semidefinite linear systems by iteration, J. SIAM Numer. Anal. 2 (1965), 281–290.
C. Lanozos,Solution of linear equations by minimized iteration, Journal of Research of the National Bureau of Standards 49 Sect. B, (1952), 33–53.
P. Läuchli, Iterative Lösung und Fehlerabschätzung in der Ausgleichsrechnung, Z. für angew. Math. und Physik 10 (1959), 245–280.
C. C. Paige and M. A. Saunders,Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal. 12 (1975), 617–629.
W. Peters, Lösung linearer Gleichungssysteme durch Projektion auf Schnitträume von Hyperebenen und Berechnung einer verallgemeinerten Inversen, Beiträge zur Numerischen Mathematik 5 (1976), 129–146.
R. J. Plemmons,Stationary iterative methods for linear systems with non-Hermitian singular matrices, Report from Dept. of Comp. Science, The University of Tennessee, Knoxville, Tennessee.
J. K. Reid,On the method of conjugate gradients for the solution of large sparse systems of linear equations, inLarge Sparse Sets of Linear Equations, ed. J. K. Reid, Academic Press, New York (1971).
H. Rutishauser,Theory of gradient methods, inRefined Iterative Methods for Computation of the Solution and the Eigenvalues of Self-Adjoint Boundary Value Problems, M. Engeli et al., Birkhäuser, Basel (1959).
H. R. Schwarz, Die Methode der konjugierten Gradienten in der Ausgleichsrechnung, Zeitschrift für Vermessungswesen 95 (1970), 130–140.
E. Stiefel, Ausgleichung ohne Aufstellung der Gausschen Normalgleichungen, Wiss. Z. Technische Hochschule Dresden 2 (1952/3), 441–442.
G. W. Stewart,Introduction to Matrix Computation, Academic Press, New York (1973).
K. Tanabe,Projection method for solving a singular system of linear equations and its application, Numer. Math.17 (1971), 203–217.
A. van der Sluis,Condition numbers and equilibration of matrices, Numer. Math. 14 (1969), 14–23.
T. M. Whitney and R. K. Meany,Two algorithms related to the method of steepest descent, SIAM J. Numer. Anal. 4 (1967), 109–118.
H. Wozniakowski,Numerical stability of the Chebyshev method for the solution of large linear systems, Numer. Math. (1977), 191–209.
D. M. Young,Iterative solution of large linear systems, Academic Press, New York (1971).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Björck, Å., Elfving, T. Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations. BIT 19, 145–163 (1979). https://doi.org/10.1007/BF01930845
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01930845