Abstract
The LSQR iterative method for solving least-squares problems may require many iterations to determine an approximate solution with desired accuracy. This often depends on the fact that singular vector components of the solution associated with small singular values of the matrix require many iterations to be determined. Augmentation of Krylov subspaces with harmonic Ritz vectors often makes it possible to determine the singular vectors associated with small singular values with fewer iterations than without augmentation. This paper describes how Krylov subspaces generated by the LSQR iterative method can be conveniently augmented with harmonic Ritz vectors. Computed examples illustrate the competitiveness of the augmented LSQR method proposed.
Similar content being viewed by others
References
Baglama, J., Reichel, L.: Augmented implicitly restarted Lanczos bidiagonalization methods. SIAM J. Sci. Comput. 27, 19–42 (2005)
Baglama, J., Reichel, L.: Restarted block Lanczos bidiagonalization methods. Numer. Algorithms 43, 251–272 (2006)
Baglama, J., Reichel, L.: An implicitly restarted block Lanczos bidiagonalization method using Leja shifts. BIT Numer. Math. (in press)
Benzi, M., Tuma, M.: A robust preconditioner with low memory requirements for large sparse least squares problems. SIAM J. Sci. Comput. 25, 499–512 (2003)
Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Björck, Å., Yuan, J.Y.: Preconditioners for least squares problems by LU factorization. Electron. Trans. Numer. Anal. 8, 26–35 (1997)
Boisvert, R., Pozo, R., Remington, K., Miller, B., Lipman, R.: MatrixMarket. The matrices are available at http://math.nist.gov/MatrixMarket/ (1996)
Choi, S.-C.: Iterative methods for singular linear equations and least squares. Ph.D. thesis, Institute for Computational and Mathematical Engineering, Stanford University (2006)
Duff, I.S., Grimes, R.G., Lewis, J.G.: User’s guide for the Harwell–Boeing sparse matrix collection (release I), France (1992). Matrices available at http://math.nist.bov/MatrixMarket/
Fong, D.C.-L., Saunders, M.A.: LSMR: an iterative algorithm for sparse least-squares problems. SIAM J. Sci. Comput. 33, 2950–2971 (2011)
Hayami, K., Yin, J.-F., Ito, T.: GMRES methods for least squares problems. SIAM J. Matrix Anal. Appl. 31, 2400–2430 (2010)
Hochstenbach, M.E.: Harmonic and refined extraction methods for the singular value problem, with applications in least squares problems. BIT Numer. Math. 44, 721–754 (2004)
Jia, Z.: Some properties of LSQR for large sparse linear least squares problems. J. Syst. Sci. Complex. 23, 815–821 (2010)
Jia, Z., Niu, D.: An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition. SIAM J. Matrix Anal. Appl. 25, 246–265 (2003)
Jia, Z., Niu, D.: A refined harmonic Lanczos bidiagonalization method and an implicitly restarted algorithm for computing the smallest singular triplets of large matrices. SIAM J. Sci. Comput. 32, 714–744 (2010)
Karimi, S., Salkuyeh, D.K., Toutounian, F.: A preconditioner for the LSQR algorithm. J. Appl. Math. Informatics 26(1–2), 213–222 (2008)
Kokiopoulou, E., Bekas, C., Gallopoulos, E.: Computing smallest singular triplets with implicitly restarted Lanczos bidiagonalization. Appl. Numer. Math. 49, 39–61 (2004)
Larsen, R.M.: Lanczos bidiagonalization with partial reorthogonalization. Ph.D. thesis, Dept. Computer Science, University of Aarhus, Aarhus, Denmark (1998)
Larsen, R.M.: Combining implicit restarts and partial reorthogonalization in Lanczos bidiagonalization (2001). http://soi.stanford.edu/~rmunk/PROPACK/
Morgan, R.B.: Computing interior eigenvalues of large matrices. Linear Algebra Appl. 154–156, 289–309 (1991)
Morgan, R.B.: A restarted GMRES method augmented with eigenvectors. SIAM J. Matrix Anal. Appl. 16, 1154–1171 (1995)
Morgan, R.B.: Implicitly restarted GMRES and Arnoldi methods for nonsymmetric systems of equations. SIAM J. Matrix Anal. Appl. 21, 1112–1135 (2000)
Morgan, R.B.: GMRES with deflated restarting. SIAM J. Sci. Comput. 24, 20–37 (2002)
Paige, C.C.: Bidiagonalization of matrices and solution of linear equations. SIAM J. Numer. Anal. 11, 197–209 (1974)
Paige, C.C., Parlett, B.N., van der Vorst, H.A.: Approximate solutions and eigenvalue bounds from Krylov subspaces. Numer. Linear Algebra Appl. 2, 115–134 (1995)
Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8, 43–71 (1982)
Reichel, L., Ye, Q.: A generalized LSQR algorithm. Numer. Linear Algebra Appl. 15, 643–660 (2008)
Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)
Simon, H.D., Zha, H.: Low rank matrix approximation using the Lanczos bidiagonalization process with applications. SIAM J. Sci. Comput. 21, 2257–2274 (2000)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Baglama, J., Reichel, L. & Richmond, D. An augmented LSQR method. Numer Algor 64, 263–293 (2013). https://doi.org/10.1007/s11075-012-9665-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-012-9665-8
Keywords
- Partial singular value decomposition
- Iterative method
- Large-scale computation
- Least-squares approximation
- LSQR
- Precondition
- Krylov subspace
- Augmentation