Skip to main content
SpringerLink
Log in

GMRES with multiple preconditioners

  • Published:
SeMA Journal Aims and scope Submit manuscript

Abstract

We propose a variant of GMRES, where multiple (two or more) preconditioners are applied simultaneously, while maintaining minimal residual optimality properties. To accomplish this, a block version of Flexible GMRES is used, but instead of considering blocks associated with multiple right hand sides, we consider a single right-hand side and grow the space by applying each of the preconditioners to all current search directions, minimizing the residual norm over the resulting larger subspace. To alleviate the difficulty of rapidly increasing storage requirements, we present a heuristic limited-memory selective algorithm, and demonstrate the effectiveness of this approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. As with GMRES, other implementations are possible, e.g., using Householder transformations.

  2. http://www.mathworks.com/matlabcentral/fileexchange/34562-multi-preconditioned-gmres

  3. These experiments were ran on a machine with an Intel Core i5-2500S CPU @ 2.70GHz with 8GB RAM.

  4. These experiments were ran on a two-socket machine, each with Intel Xeon CPU E5-2687W 0 @ 3.10 GHz (i.e. \(2\times 8\) cores), and with 64GB memory total.

References

  1. Ayuso de Dios, B., Barker, A.T., Vassilevski, P.S.: A combined preconditioning strategy for nonsymmetric systems. SIAM J. Sci. Comput. 36, A2533G–A2556 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bakhos, T., Ladenheim, S., Kitanidis, P.K., Saibaba, A.K., Szyld D.B.: Multipreconditioned GMRES for shifted systems. In: Research Report (2016), Department of Mathematics, Temple University (2016). Accessed 31 Mar 2016

  3. Bai, Z.-Z.: Block preconditioners for elliptic PDE-constrained optimization problems. Computing 91, 379–395 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Benzi, M.: Preconditioning techniques for large linear systems: a survey. J. Comput. Phys. 182, 418–477 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  5. Birkhof, G., Varga, R.S., Young Jr., D.M.: Alternating direction implicit methods. Adv. Comput. 3, 189–273 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bridson, R., Greif, C.: A multipreconditioned conjugate gradient algorithm. SIAM J. Matrix Anal. Appl. 27, 1056–1068 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Calandra, H., Gratton, S., Langou, J., Pinel, X., Vasseur, X.: Flexible variants of block restarted GMRES methods with application to geophysics. SIAM J. Sci. Comput. 34, A714–A736 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. de Sturler, E.: Nested Krylov methods based on GCR. J. Comput. Appl. Math. 67, 15–41 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  9. Elbouyahyaoui, L., Messaoudi, A., Sadok, H.: Algebraic properties of the block GMRES and block Arnoldi methods. Electr. Trans. Num. Anal. 33, 207–220 (2008–2009)

  10. Elman, H., Howle, V.E., Shadid, J., Shuttleworth, R., Tuminaro, R.: Block preconditioners based on approximate commutators. SIAM J. Sci. Comput. 27, 1651–1668 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  11. Elman, H., Silvester, D., Wathen, A.: Finite elements and fast iterative solvers, Second Edition, Oxford University Press, Oxford (2014)

  12. Freund, R.W.: Model reduction methods based on Krylov subspaces. Acta Num. 12, 267–319 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gaul, A., Gutknecht, M.H., Liesen, J., Nabben, R.: A framework for deflated and augmented Krylov subspace methods. SIAM J. Matrix Anal. Appl. 34, 495–518

  14. Greenbaum, A., Pták, V., Strakoš, Z.: Any nonincreasing convergence curve is possible for GMRES. SIAM J. Matrix Anal. Appl. 17, 465–469 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  15. Greif, C., Rees, T., Szyld, D.B.: Additive Schwarz with variable weights, domain decomposition methods in science and engineering XXI. In: Erhel, J., Gander, M., Halpern, L., Pichot, G., Sassi, T., Widlund, O. (eds.) Proceedings of the 21st international conference on domain decomposition methods. Lecture Notes in Computer Science and Engineering, vol. 98, pp. 655–662. Springer, Berlin (2014)

    Google Scholar 

  16. Gutknecht, M.H.: Block Krylov space methods for linear systems with multiple right-hand sides: an introduction. In: Siddiqi, A.H., Duff, I.S., Christensen, O. (eds.) Modern mathematical models, methods and algorithms for real world systems, pp. 420–447. Anamaya, New Delhi (2007)

  17. Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE constraints. Mathematical modelling: theory and applications. Springer, New York (2008)

  18. HSL (2013). A collection of Fortran codes for large scale scientific computation. http://www.hsl.rl.ac.uk

  19. Johnsson, S.L., Saad, Y., Schultz, M.H.: Alternating direction methods on multiprocessors. SIAM J. Stat. Sci. Comput. 8, 686–700 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  20. Kay, D., Loghin, D., Wathen, A.J.: A preconditioner for the steady-state Navier–Stokes equations. SIAM J. Sci. Comput. 24, 237–256 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  21. Keller, C., Gould, N.I.M., Wathen, A.J.: Constraint preconditioning for indefinite linear systems. SIAM J. Matrix Anal. Appl. 21, 1300–1317 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  22. Langou, J.: Iterative methods for solving linear systems with multiple right hand sides. Ph.D. thesis, INSA, Toulouse, June 2003. CERFACS Report TH/PA/03/24 (2003)

  23. Lukšan, L., Vlček, J.: Indefinitely preconditioned inexact Newton method for large sparse equality constrained nonlinear programming problems. Num. Linear Algebra Appl. 5, 219–247 (1998)

    Article  MATH  Google Scholar 

  24. O’Leary, D.P.: The block conjugate gradient algorithm and related methods. Linear Algebra Appl. 29, 293–322 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  25. O’Leary, D.P., White, R.E.: Multi-splittings of matrices and parallel solution of linear systems. SIAM J. Algebraic Discret. Methods 6, 630–640 (1985)

    Article  MATH  Google Scholar 

  26. Pearson, J.W., Wathen, A.J.: A new approximation of the Schur complement in preconditioners for PDE-constrained optimization. Linear Algebra Appl. 19, 816–829 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  27. Pestana, J.: Nonstandard inner products and preconditioned iterative methods. D.Phil. thesis, University of Oxford (2011)

  28. Pestana, J., Wathen, A.J.: Combination preconditioning of saddle point systems for positive definiteness. Num. Linear Algebra Appl. 20, 785–808 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  29. Rees, T., Dollar, H.S., Wathen, A.J.: Optimal solvers for PDE-constrained optimization. SIAM J. Sci. Comput. 32, 271–298 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  30. Robbé, M., Sadkane, M.: Exact and inexact breakdowns in the block GMRES method. Linear Algebra Appl. 419, 265–285 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  31. Ruhe, A.: Implementation aspects of band Lanczos algorithms for computation of eigenvalues of large sparse symmetric matrices. Math. Comput. 33, 680–687 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  32. Rui, P.-L., Yong, H., Chen, R.-S.: Multipreconditioned GMRES method for electromagnetic wave scattering problems. Microw. Opt. Technol. Lett. 50, 150–152 (2008)

    Article  Google Scholar 

  33. Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Comput. 14, 461–469 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  34. Saad, Y.: Iterative methods for sparse linear systems, 2nd edn. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  35. Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  36. Sadkane, M.: A block Arnoldi-Chebyshev method for computing the leading eigenpairs of large sparse unsymmetric matrices. Num. Math. 23, 181–193 (2009)

    MathSciNet  MATH  Google Scholar 

  37. Sadkane, M.: Block Arnoldi and Davidson methods for unsymmetric large eigenvalue problems. Num. Math. 64, 687–706 (1993)

    MathSciNet  MATH  Google Scholar 

  38. Sarkis, M., Szyld, D.B.: Optimal left and right additive Schwarz preconditioning for minimal residual methods with Euclidean and energy norms. Comput. Methods Appl. Mech. Eng. 196, 1612–1621 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  39. Schöberl, J., Zulehner, W.: Symmetric indefinite preconditioners for saddle point problems with applications to PDE-constrained optimization problems. SIAM J. Matrix Anal. Appl. 29, 752–773 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  40. Silvester, D., Elman, H., Kay, D., Wathen, A.: Efficient preconditioning of the linearized Navier-Stokes equations for incompressible flow. J. Comput. Appl. Math. 128, 261–279 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  41. Simoncini, V., Gallopouolos, E.: Convergence properties of block GMRES and matrix polynomials. Linear Algebra Appl. 47, 97–119 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  42. Simoncini, V., Szyld, D.B.: On the occurrence of superlinear convergence of exact and inexact Krylov subspace methods. SIAM Rev. 47, 247–272 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  43. Simoncini, V., Szyld, D.B.: Recent computational developments in Krylov subspace methods for linear systems. Num. Linear Algebra Appl. 14, 1–59 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  44. Sleijpen, G.L., van der Vorst, H.A., Modersitzki, J.: Differences in the effects of rounding errors in Krylov solvers for symmetric indefinite linear systems. SIAM J. Matrix Anal. Appl. 22, 726–751 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  45. Stoll, M., Wathen, A.J.: Combination preconditioning and the Bramble-Pasciak\(^{+}\) preconditioner. SIAM J. Matrix Anal. Appl. 30, 582–608 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  46. Tröltzsch, F.: Optimal control of partial differential equations: Theory, methods and applications. American Mathematical Society, Providence (2010)

    Book  MATH  Google Scholar 

  47. Vital, B.: Etude de quelques méthodes de résolution de problèmes linéaires de grande taille sur multiprocessor. Ph. D. thesis, Université de Rennes (1990)

Download references

Acknowledgments

We thank the two anonymous referees for their questions and comments, which helped us improve our presentation.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of British Columbia, Vancouver, British Columbia, V6T 1Z4, Canada

    Chen Greif

  2. Department of Scientific Computing, STFC Rutherford Appleton Laboratory, Chilton Didcot, Oxfordshire, OX11 0QX, UK

    Tyrone Rees

  3. Department of Mathematics, Temple University (038-16), 1805 N. Broad Street, Philadelphia, PA, 19122-6094, USA

    Daniel B. Szyld

Authors
  1. Chen Greif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tyrone Rees
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel B. Szyld
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel B. Szyld.

Additional information

The work of Chen Greif was supported in part by Discovery Grant 261539 from the Natural Sciences and Engineering Research Council of Canada (NSERC). The work of Tyrone Rees was supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC). The work of Daniel B. Szyld was supported in part by the U.S. National Science Foundation under grant DMS-1418882.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Greif, C., Rees, T. & Szyld, D.B. GMRES with multiple preconditioners. SeMA 74, 213–231 (2017). https://doi.org/10.1007/s40324-016-0088-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40324-016-0088-7

Keywords

Mathematics Subject Classification

Search

Navigation