Generalized and Sparse Least Squares Problems
Least squares problems arise frequently in optimization, e.g., in interior point methods. This paper surveys methods for solving least squares problems of nonstandard form such as generalized and sparse problems. Algorithms for standard and banded problems are first studied. Methods for solving generalized least squares problems are then surveyed. The special case of weighted problems is treated in detail. Iterative refinement is discussed as a general technique for improving the accuracy of computed solutions. Least squares problems where the solution is constrained by linear equality constraints or quadratic constraints are also treated.
Graph theoretic methods for reordering rows and columns to reduce fill in when solving sparse least squares problems are surveyed. The numerical phase of sparse Cholesky and sparse QR factorization is then discussed. In particular the multifrontal method, which currently is the most efficient implementation, is described.
KeywordsNormal Equation Interior Point Method Cholesky Factor Iterative Refinement Triangular System
Unable to display preview. Download preview PDF.
- C. Ashcraft. A vector implementation of the multifrontal method for large sparse positive definite systems. Technical Report ETA-TR-51, Engineering Technology Division, Boeing Computer Services, 1987.Google Scholar
- Å. Björck. Least squares methods. In R G. Ciarlet and J. L. Lions, editors,Handbook of Numerical Analysis, volume I: Finite Difference Methods-Solution of Equations in R n, pages 466–647. Elsevier/North Holland, Amsterdam, 1990.Google Scholar
- Å. Björck. Algorithms for linear least squares problems. In E. Spedicato, editor,Computer Algorithms for Solving Linear Algebraic Equations; The State of the Art., NATO ASI Series F: Computer and Systems Sciences, Vol. 77, pages 57–92, Berlin, 1991. Springer-Verlag.Google Scholar
- Å. Björck. Pivoting and stability in the augmented system method. In D. F. Griffiths and G. A. Watson, editors,Numerical Analysis 1991: Proceedings of the 14th Dundee Conference, June 1991, Pitman Research Notes in Mathematics 260, pages 1–16. Longman Scientific and Technical, 1992.Google Scholar
- Å. Björck. Numerics of Gram-Schmidt orthogonalization. Linear Algebra and Appl., 197: to appear, 1994.Google Scholar
- I. S. Duff and J. K. Reid. MA27—a set of Fortran subroutines for solving sparse symmetric sets of linear equations. Technical Report R. 10533, AERE, Harwell, England, 1982.Google Scholar
- L. V. Foster. Modifications of the normal equations method that are numerically stable. In G. H. Golub and P. Van Dooren, editors,Numerical Linear Algebra, Digital Signal Processing and Parallel Algorithms, NATO ASI Series, pages 501— 512, Berlin, 1991. Springer-Verlag.Google Scholar
- P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright. A Schur-complement method for sparse quadratic programming. In M.G. Cox and S. Hammarling, editors,Reliable Numerical Computation, pages 113–138, Oxford, 1990. Clarendon Press.Google Scholar
- Ö. Leringe and P.-À. Wedin. A comparison between different methods to compute a vector xwhich minimizes ∥Ax — b∥ 2 when Gx = h. Technical Report, Department of Computer Science, Lund University, 1970.Google Scholar
- J. G. Lewis, D. J. Pierce, and D. C. Wah. A multifrontal Householder QR factorization. Technical Report ECA-TR-127, Engineering and Scientific Services Division, Boeing Computer Services, 1989.Google Scholar
- P. Matstoms. The Multifrontal Solution of Sparse Linear Least Squares Problems. Lie. Thesis, Linköping University, 1991.Google Scholar
- P. Matstoms. Sparse QR factorization in MATLAB. ACM Trans. Math. Software, to appear, 1994.Google Scholar
- M. J. D. Powell and J. K. Reid. On applying Householder’s method to linear least squares problems. In A. J. M. Morell, editor,Proceedings of the IFIP Congress 68, pages 122–126, Amsterdam, 1965. North Holland.Google Scholar
- C. F. Van Loan. Computing the CS and the generalized singular value decomposition. Numtr. Math., 46: 479–492, 1985.Google Scholar