Numerische Mathematik

, Volume 55, Issue 6, pp 667–684

On the augmented system approach to sparse least-squares problems

  • M. Arioli
  • I. S. Duff
  • P. P. M. de Rijk


We study the augmented system approach for the solution of sparse linear least-squares problems. It is well known that this method has better numerical properties than the method based on the normal equations. We use recent work by Arioli et al. (1988) to introduce error bounds and estimates for the components of the solution of the augmented system. In particular, we find that, using iterative refinement, we obtain a very robust algorithm and our estimates of the error are accurate and cheap to compute. The final error and all our error estimates are much better than the classical or Skeel's error analysis (1979) indicates. Moreover, we prove that our error estimates are independent of the row scaling of the augmented system and we analyze the influence of the Björck scaling (1967) on these estimates. We illustrate this with runs both on large-scale practical problems and contrived examples, comparing the numerical behaviour of the augmented systems approach with a code using the normal equations. These experiments show that while the augmented system approach with iterative refinement can sometimes be less efficient than the normal equations approach, it is comparable or better when the least-squares matrix has a full row, and is, in any case, much more stable and robust.

Subject Classifications

AMS(MOS): 65F20 CR: G1.3 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Arioli, M., Demmel, J.W., Duff, I.S.: Solving sparse linear systems with sparse backward error. Report CSS 214, CSS Division, Harwell Laboratory, England. SIAM J. Matrix Anal. Appl. 1988 (to appear)Google Scholar
  2. Bartels, R.H., Golub, G.H., Saunders, M.A.: Numerical techniques in mathematical programming. In: Rosen, J.B., Mangasarian, O.L., Ritter, K. (eds.): Nonlinear programming, pp. 123–176. New York: Academic Press 1970Google Scholar
  3. Björck, Å.: Iterative refinement of linear least squares solutions. BIT7, 257–278 (1967)Google Scholar
  4. Duff, I.S., Grimes, R.G., Lewis, J.G.: Sparse matrix test problems. Report CSS 191, CSS Division, Harwell Laboratory, England. ACM Trans. Math. Software 1987 (to appear)Google Scholar
  5. Duff, I.S., Reid, J.K.: The multifrontal solution of indefinite sparse symmetric linear equations. ACM Trans. Math. Software9, 302–325 (1983)CrossRefGoogle Scholar
  6. Gay, D.M.: Electronic mail distribution of linear programming test problems. Mathematical Programming Society COAL Newsletter 1985Google Scholar
  7. Golub, G.H., Van Loan, C.F.: Matrix computations. 1st Ed. North Oxford Academic, Oxford, and John Hopkins Press, Baltimore 1983Google Scholar
  8. Hachtel, G.D.: Extended applications of the sparse tableau approach—finite elements and least squares. In: Spillers, W.R. (ed.): Basic question of design theory. Amsterdam: North Holland 1974Google Scholar
  9. Hager, W.W.: Condition estimators. SIAM J. Sci. Stat. Comput.5, 311–316 (1984)CrossRefGoogle Scholar
  10. Higham, N.J.: A survey of condition number estimation for triangular matrices. SIAM Review29, 575–596 (1987)CrossRefGoogle Scholar
  11. Läuchli, P.: Jordan-Elimination und Ausgleichung nach kleinsten Quadraten. Numer. Math.3, 226–240 (1961)CrossRefGoogle Scholar
  12. Karmarkar, N.: A new polynomial time algorithm for linear programming. Combinatorica4, 373–395 (1984)Google Scholar
  13. Oettli, W., Prager, W.: Compatibility of approximate solution of linear equations with given error bounds for coefficients and right-hand sides. Numer. Math.6, 405–409 (1964)CrossRefGoogle Scholar
  14. Skeel, R.D.: Scaling for numerical stability in Gaussian elimination. J. ACM26, 494–526 (1979)CrossRefGoogle Scholar
  15. Skeel, R.D.: Iterative refinement implies numerical stability for Gaussian elimination. Math. Comput.35, 817–832 (1980)Google Scholar
  16. Van Loan, C.F.: On the method of weighting for equality-constrained least-squares problems. SIAM J. Numer. Anal.22, 851–864 (1985)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 1989

Authors and Affiliations

  • M. Arioli
    • 1
  • I. S. Duff
    • 1
  • P. P. M. de Rijk
    • 1
  1. 1.Computer Science and Systems DivisionHarwell LaboratoryDidcotUK

Personalised recommendations