Skip to main content
SpringerLink
Log in

Randomized flexible GMRES with deflated restarting

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

For a high dimensional problem, a randomized Gram-Schmidt (RGS) algorithm is beneficial in computational costs as well as numerical stability. We apply this dimension reduction technique by random sketching to Krylov subspace methods, e.g. to the generalized minimal residual method (GMRES). We propose a flexible variant of GMRES with the randomized Gram-Schmidt–based Arnoldi iteration to produce a set of basis vectors of the Krylov subspace. Even though the Krylov basis is no longer \(l_2\) orthonormal, its random projection onto the low dimensional space achieves \(l_2\) orthogonality. As a result, the numerical stability is observed which turns out to be independent of the dimension of the problem even in extreme scale problems. On the other hand, as the harmonic Ritz values are commonly used in GMRES with deflated restarting to improve convergence, we consider another deflation strategy, for instance disregarding the singular vectors associated with the smallest singular values. We thus introduce a new algorithm of the randomized flexible GMRES with singular value decomposition (SVD)–based deflated restarting. At the end, we carry out numerical experiments in the context of compressible turbulent flow simulations. Our proposed approach exhibits a quite competitive numerical behaviour to existing methods while reducing computational costs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The authors confirm that the data supporting the findings of this study are available within the article.

Code Availability

All code and scripts to reproduce plots can be found at Jang’s GitHub (https://github.com/Yongseok7717/RandomizedGMRES).

References

  1. Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)

    Article  MathSciNet  Google Scholar 

  2. Paige, C.C., Saunders, M.A.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975)

    Article  MathSciNet  Google Scholar 

  3. Nachtigal, N.M., Reddy, S.C., Trefethen, L.N.: How fast are nonsymmetric matrix iterations? SIAM J. Matrix Anal. Appl. 13(3), 778–795 (1992)

    Article  MathSciNet  Google Scholar 

  4. Embree, M.: How descriptive are GMRES convergence bounds? arXiv:2209.01231 (2022)

  5. Morgan, R.B.: GMRES with deflated restarting. SIAM J. Sci. Comput. 24(1), 20–37 (2002)

    Article  MathSciNet  Google Scholar 

  6. Giraud, L., Gratton, S., Pinel, X., Vasseur, X.: Flexible GMRES with deflated restarting. SIAM J. Sci. Comput. 32(4), 1858–1878 (2010)

    Article  MathSciNet  Google Scholar 

  7. Röllin, S., Fichtner, W.: Improving the accuracy of GMRES with deflated restarting. SIAM J. Sci. Comput. 30(1), 232–245 (2008)

    Article  MathSciNet  Google Scholar 

  8. Simoncini, V.: On the convergence of restarted Krylov subspace methods. SIAM J. Matrix Anal. Appl. 22(2), 430–452 (2000)

    Article  MathSciNet  Google Scholar 

  9. Parks, M.L., De Sturler, E., Mackey, G., Johnson, D.D., Maiti, S.: Recycling Krylov subspaces for sequences of linear systems. SIAM J. Sci. Comput. 28(5), 1651–1674 (2006)

    Article  MathSciNet  Google Scholar 

  10. Daas, H.A., Grigori, L., Hénon, P., Ricoux, P.: Recycling Krylov subspaces and truncating deflation subspaces for solving sequence of linear systems. ACM Trans. Math. Softw. (TOMS) 47(2), 1–30 (2021)

    Article  MathSciNet  Google Scholar 

  11. Carvalho, L.M., Gratton, S., Lago, R., Vasseur, X.: A flexible generalized conjugate residual method with inner orthogonalization and deflated restarting. SIAM J. Matrix Anal. Appl. 32(4), 1212–1235 (2011)

    Article  MathSciNet  Google Scholar 

  12. Balabanov, O., Grigori, L.: Randomized Gram-Schmidt process with application to GMRES. SIAM J. Sci. Comput. 44(3), 1450–1474 (2022)

    Article  MathSciNet  Google Scholar 

  13. Balabanov, O., Grigori, L.: Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems. arXiv:2111.14641 (2021)

  14. Nakatsukasa, Y., Tropp, J.A.: Fast & accurate randomized algorithms for linear systems and eigenvalue problems. arXiv:2111.00113 (2021)

  15. Bjorck, A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia, PA (1996)

    Book  Google Scholar 

  16. Achlioptas, D.: Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66(4), 671–687 (2003)

    Article  MathSciNet  Google Scholar 

  17. Halko, N., Martinsson, P.-G., Tropp, J.A.: Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53(2), 217–288 (2011)

    Article  MathSciNet  Google Scholar 

  18. Woodruff, D.P.: Sketching as a tool for numerical linear algebra. Found. Trends Theor. Comput. Sci. 10(1–2), 1–157 (2014)

    Article  MathSciNet  Google Scholar 

  19. Balabanov, O., Nouy, A.: Randomized linear algebra for model reduction. Part I: Galerkin methods and error estimation. Adv. Comput. Math. 45(5), 2969–3019 (2019)

    Article  MathSciNet  Google Scholar 

  20. Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Comput. 14(2), 461–469 (1993)

    Article  MathSciNet  Google Scholar 

  21. Paige, C.C., Parlett, B.N., Vorst, H.A.: Approximate solutions and eigenvalue bounds from Krylov subspaces. Numer. Linear Algebra Appl. 2(2), 115–133 (1995)

    Article  MathSciNet  Google Scholar 

  22. Gaul, A., Gutknecht, M.H., Liesen, J., Nabben, R.: A framework for deflated and augmented Krylov subspace methods. SIAM J. Matrix Anal. Appl. 34(2), 495–518 (2013)

    Article  MathSciNet  Google Scholar 

  23. Giraud, L., Langou, J., Rozloznik, M.: The loss of orthogonality in the Gram-Schmidt orthogonalization process. Comput. Math. Appl. 50(7), 1069–1075 (2005)

    Article  MathSciNet  Google Scholar 

  24. Abdelmalek, N.N.: Round off error analysis for Gram-Schmidt method and solution of linear least squares problems. BIT Numer. Math. 11(4), 345–367 (1971)

    Article  MathSciNet  Google Scholar 

  25. Giraud, L., Langou, J., Rozložník, M., Eshof, Jvd: Rounding error analysis of the classical Gram-Schmidt orthogonalization process. Numerische Mathematik 101(1), 87–100 (2005)

    Article  MathSciNet  Google Scholar 

  26. Björck, Å.: Solving linear least squares problems by Gram-Schmidt orthogonalization. BIT Numer. Math. 7(1), 1–21 (1967)

    Article  MathSciNet  Google Scholar 

  27. Van Loan, C.F., Golub, G.: Matrix computations (Johns Hopkins studies in mathematical sciences). Matrix Computations (1996)

  28. Tropp, J.A.: Improved analysis of the subsampled randomized Hadamard transform. Adv. Adapt. Data Anal. 3(01n02), 115–126 (2011)

    Article  MathSciNet  Google Scholar 

  29. Wang, S.: A practical guide to randomized matrix computations with MATLAB implementations. arXiv:1505.07570 (2015)

  30. Jacobs, E.N., Ward, K.E., Pinkerton, R.M.: The Characteristics of 78 Related Airfoil Section from Tests in the Variable-Density Wind Tunnel. Technical report, NASA (1933)

    Google Scholar 

  31. Allmaras, S.R., Johnson, F.T.: Modifications and clarifications for the implementation of the Spalart-Allmaras turbulence model. In: Seventh International Conference on Computational Fluid Dynamics (ICCFD7), vol. 1902, pp. 1–11 (2012). XBig Island, HI

  32. Crivellini, A., Bassi, F.: An implicit matrix-free discontinuous Galerkin solver for viscous and turbulent aerodynamic simulations. Comput. Fluids 50(1), 81–93 (2011)

    Article  MathSciNet  Google Scholar 

  33. Cambier, L., Heib, S., Plot, S.: The ONERA elsA CFD software: input from research and feedback from industry. Mech. Ind., (2013)

  34. Baker, A.H., Jessup, E.R., Kolev, T.V.: A simple strategy for varying the restart parameter in GMRES(m). J. Comput. Appl. Math. 230(2), 751–761 (2009)

    Article  MathSciNet  Google Scholar 

  35. Arts, T., De Rouvroit, M.L.: Aero-thermal performance of a two dimensional highly loaded transonic turbine nozzle guide vane: a test case for inviscid and viscous flow computations. In: Turbo Expo: Power for Land, Sea, and Air, vol. 79047, pp. 001–01106 (1990). American Society of Mechanical Engineers

  36. Arts, T., De Rouvroit, M.L., Rutherford, A.: Aero-thermal Investigation of a Highly Loaded Transonic Linear Turbine Guide Vane Cascade. von Karman Institute for Fluid Dynamics, Technical report (1990)

    Google Scholar 

Download references

Funding

This work is supported by DGAC (Direction Générale de l’Aviation Civile) in the frame of the SONICE project.

Author information

Author notes

  1. Laura Grigori

    Present address: EPFL, Lausanne, Switzerland

Authors and Affiliations

  1. ONERA, Université Paris-Saclay, Châtillon, France

    Yongseok Jang, Emeric Martin & Cédric Content

  2. Inria, Sorbonne Université, CNRS LJLL, Paris, France

    Laura Grigori

Authors
  1. Yongseok Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura Grigori
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emeric Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cédric Content
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y. Jang: writing — original draft, review and editing, investigation, methodology, visualization, software. L. Grigori: conceptualization, formal analysis, supervision. E. Martin: review and editing. C. Content: data curation, project administration.

Corresponding author

Correspondence to Yongseok Jang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jang, Y., Grigori, L., Martin, E. et al. Randomized flexible GMRES with deflated restarting. Numer Algor (2024). https://doi.org/10.1007/s11075-024-01801-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11075-024-01801-3

Keywords

Mathematics Subject Classification (2010)

Search

Navigation