Variants of residual smoothing with a small residual gap

  • Kensuke AiharaEmail author
  • Ryosuke Komeyama
  • Emiko Ishiwata


Short-recurrence Krylov subspace methods, such as conjugate gradient squared-type methods, often exhibit large oscillations in the residual norms, leading to a large residual gap and a loss of attainable accuracy for the approximate solutions. Residual smoothing is useful for obtaining smooth convergence for the residual norms, but it has been shown that this does not improve the maximum attainable accuracy in most cases. In the present study, we reformulate the smoothing scheme from a novel perspective. The smoothed sequences do not usually affect the primary sequences in conventional smoothing schemes. In contrast, we design a variant of residual smoothing in which the primary and smoothed sequences influence each other. This approach enables us to avoid the propagation of large rounding errors, and results in a smaller residual gap, and thus a higher attainable accuracy. We present a rounding error analysis and numerical experiments to demonstrate the effectiveness of our proposed smoothing scheme.


Linear system Krylov subspace method CGS-type method Residual smoothing Residual gap 

Mathematics Subject Classification




The authors would like to thank the reviewers for their constructive comments and helpful suggestions. The first author is most grateful to Dr. Kuniyoshi Abe (Gifu Shotoku University) and Dr. Gerard L.G. Sleijpen (Utrecht University) for their guidance and invaluable discussions when the author was a Ph.D. student. The present study was supported by JSPS KAKENHI Grant Number JP18K18064.


  1. 1.
    Aihara, K.: Variants of the groupwise update strategy for short-recurrence Krylov subspace methods. Numer. Algorithms 75, 397–412 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Chan, T.F., Gallopoulos, E., Simoncini, V., Szeto, T., Tong, C.H.: A quasi-minimal residual variant of the Bi-CGStab algorithm for nonsymmetric systems. SIAM J. Sci. Comput. 15, 338–347 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Davis, T.A., Hu, Y.: The university of Florida sparse matrix collection. ACM Trans. Math. Softw. 38, 1–25 (2011)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Fletcher, R.: Conjugate gradient methods for indefinite systems. In: Watson, G.A. (ed.) Numerical Analysis. Lecture Notes in Math., vol. 506, pp. 73–89. Springer, Berlin (1976)Google Scholar
  5. 5.
    Fokkema, D.R., Sleijpen, G.L.G., van der Vorst, H.A.: Generalized conjugate gradient squared. J. Comput. Appl. Math. 71, 125–146 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Freund, R.W.: A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems. SIAM J. Sci. Comput. 14, 470–482 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Freund, R.W., Nachtigal, N.M.: QMR: a quasi minimal residual method for non-Hermitian linear systems. Numer. Math. 60, 315–339 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Greenbaum, A.: Estimating the attainable accuracy of recursively computed residual methods. SIAM J. Matrix Anal. Appl. 18, 535–551 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Gutknecht, M.H., Rozložník, M.: By how much can residual minimization accelerate the convergence of orthogonal residual methods? Numer. Algorithms 27, 189–213 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Gutknecht, M.H., Rozložník, M.: Residual smoothing techniques: Do they improve the limiting accuracy of iterative solvers? BIT 41, 86–114 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Paige, C.C., Saunders, M.A.: Solutions of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12, 617–629 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)CrossRefzbMATHGoogle Scholar
  13. 13.
    Schönauer, W.: Scientific Computing on Vector Computers. Elsevier, Amsterdam (1987)Google Scholar
  14. 14.
    Sleijpen, G.L.G., van der Vorst, H.A.: Reliable updated residuals in hybrid Bi-CG methods. Computing 56, 141–163 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Sonneveld, P.: CGS, a fast Lanczos-type solver for nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 10, 36–52 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    van den Eshof, J., Sleijpen, G.L.G.: Inexact Krylov subspace methods for linear systems. SIAM J. Matrix Anal. Appl. 26, 125–153 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    van der Vorst, H.A.: Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 13, 631–644 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    van der Vorst, H.A., Ye, Q.: Residual replacement strategies for Krylov subspace iterative methods for the convergence of true residuals. SIAM J. Sci. Comput. 22, 835–852 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Weiss, R.: Parameter-Free Iterative Linear Solvers. Mathematical Research, vol. 97. Akademie Verlag, Berlin (1996)zbMATHGoogle Scholar
  20. 20.
    Zhou, L., Walker, H.F.: Residual smoothing techniques for iterative methods. SIAM J. Sci. Comput. 15, 297–312 (1994)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceTokyo City UniversityTokyoJapan
  2. 2.Hitachi Industry & Control Solutions, Ltd.Hitachi-shiJapan
  3. 3.Department of Applied MathematicsTokyo University of ScienceTokyoJapan

Personalised recommendations