Preconditioned GMRES methods for least squares problems

Abstract

For least squares problems of minimizing ∥b −Ax∥₂ whereA is a large sparsem ×n (m ≥n) matrix, the common method is to apply the conjugate gradient method to the normal equationA ^T Ax =A ^T b. However, the condition number ofA ^T A is square of that ofA, and convergence becomes problematic for severely ill-conditioned problems even with preconditioning. In this paper, we propose two methods for applying the GMRES method to the least squares problem by using ann ×m matrixB. We give the necessary and sufficient condition thatB should satisfy in order that the proposed methods give a least squares solution. Then, for implementations forB, we propose an incomplete QR decomposition IMGS(l). Numerical experiments showed that the simplest casel = 0 gives the best results, and converges faster than previous methods for severely ill-conditioned problems. The preconditioner IMGS(0) is equivalent to the caseB = (diag(A ^T A))⁻¹ A ^T, so (diag(A ^T A))⁻¹ A ^T was the best preconditioner among IMGS(l) and Jennings’ IMGS(τ). On the other hand, CG-IMGS(0) was the fastest for well-conditioned problems.