SeMA Journal

, Volume 74, Issue 2, pp 213–231 | Cite as

GMRES with multiple preconditioners



We propose a variant of GMRES, where multiple (two or more) preconditioners are applied simultaneously, while maintaining minimal residual optimality properties. To accomplish this, a block version of Flexible GMRES is used, but instead of considering blocks associated with multiple right hand sides, we consider a single right-hand side and grow the space by applying each of the preconditioners to all current search directions, minimizing the residual norm over the resulting larger subspace. To alleviate the difficulty of rapidly increasing storage requirements, we present a heuristic limited-memory selective algorithm, and demonstrate the effectiveness of this approach.


Iterative methods Linear systems Preconditioning Krylov subspaces GMRES 

Mathematics Subject Classification

65F10 65N22 15A06 



We thank the two anonymous referees for their questions and comments, which helped us improve our presentation.


  1. 1.
    Ayuso de Dios, B., Barker, A.T., Vassilevski, P.S.: A combined preconditioning strategy for nonsymmetric systems. SIAM J. Sci. Comput. 36, A2533G–A2556 (2014)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Bakhos, T., Ladenheim, S., Kitanidis, P.K., Saibaba, A.K., Szyld D.B.: Multipreconditioned GMRES for shifted systems. In: Research Report (2016), Department of Mathematics, Temple University (2016). Accessed 31 Mar 2016Google Scholar
  3. 3.
    Bai, Z.-Z.: Block preconditioners for elliptic PDE-constrained optimization problems. Computing 91, 379–395 (2011)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Benzi, M.: Preconditioning techniques for large linear systems: a survey. J. Comput. Phys. 182, 418–477 (2002)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Birkhof, G., Varga, R.S., Young Jr., D.M.: Alternating direction implicit methods. Adv. Comput. 3, 189–273 (1962)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Bridson, R., Greif, C.: A multipreconditioned conjugate gradient algorithm. SIAM J. Matrix Anal. Appl. 27, 1056–1068 (2006)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Calandra, H., Gratton, S., Langou, J., Pinel, X., Vasseur, X.: Flexible variants of block restarted GMRES methods with application to geophysics. SIAM J. Sci. Comput. 34, A714–A736 (2012)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    de Sturler, E.: Nested Krylov methods based on GCR. J. Comput. Appl. Math. 67, 15–41 (1996)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Elbouyahyaoui, L., Messaoudi, A., Sadok, H.: Algebraic properties of the block GMRES and block Arnoldi methods. Electr. Trans. Num. Anal. 33, 207–220 (2008–2009)Google Scholar
  10. 10.
    Elman, H., Howle, V.E., Shadid, J., Shuttleworth, R., Tuminaro, R.: Block preconditioners based on approximate commutators. SIAM J. Sci. Comput. 27, 1651–1668 (2006)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Elman, H., Silvester, D., Wathen, A.: Finite elements and fast iterative solvers, Second Edition, Oxford University Press, Oxford (2014)Google Scholar
  12. 12.
    Freund, R.W.: Model reduction methods based on Krylov subspaces. Acta Num. 12, 267–319 (2003)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Gaul, A., Gutknecht, M.H., Liesen, J., Nabben, R.: A framework for deflated and augmented Krylov subspace methods. SIAM J. Matrix Anal. Appl. 34, 495–518Google Scholar
  14. 14.
    Greenbaum, A., Pták, V., Strakoš, Z.: Any nonincreasing convergence curve is possible for GMRES. SIAM J. Matrix Anal. Appl. 17, 465–469 (1996)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Greif, C., Rees, T., Szyld, D.B.: Additive Schwarz with variable weights, domain decomposition methods in science and engineering XXI. In: Erhel, J., Gander, M., Halpern, L., Pichot, G., Sassi, T., Widlund, O. (eds.) Proceedings of the 21st international conference on domain decomposition methods. Lecture Notes in Computer Science and Engineering, vol. 98, pp. 655–662. Springer, Berlin (2014)Google Scholar
  16. 16.
    Gutknecht, M.H.: Block Krylov space methods for linear systems with multiple right-hand sides: an introduction. In: Siddiqi, A.H., Duff, I.S., Christensen, O. (eds.) Modern mathematical models, methods and algorithms for real world systems, pp. 420–447. Anamaya, New Delhi (2007)Google Scholar
  17. 17.
    Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE constraints. Mathematical modelling: theory and applications. Springer, New York (2008)Google Scholar
  18. 18.
    HSL (2013). A collection of Fortran codes for large scale scientific computation.
  19. 19.
    Johnsson, S.L., Saad, Y., Schultz, M.H.: Alternating direction methods on multiprocessors. SIAM J. Stat. Sci. Comput. 8, 686–700 (1987)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Kay, D., Loghin, D., Wathen, A.J.: A preconditioner for the steady-state Navier–Stokes equations. SIAM J. Sci. Comput. 24, 237–256 (2002)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Keller, C., Gould, N.I.M., Wathen, A.J.: Constraint preconditioning for indefinite linear systems. SIAM J. Matrix Anal. Appl. 21, 1300–1317 (2000)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Langou, J.: Iterative methods for solving linear systems with multiple right hand sides. Ph.D. thesis, INSA, Toulouse, June 2003. CERFACS Report TH/PA/03/24 (2003)Google Scholar
  23. 23.
    Lukšan, L., Vlček, J.: Indefinitely preconditioned inexact Newton method for large sparse equality constrained nonlinear programming problems. Num. Linear Algebra Appl. 5, 219–247 (1998)CrossRefMATHGoogle Scholar
  24. 24.
    O’Leary, D.P.: The block conjugate gradient algorithm and related methods. Linear Algebra Appl. 29, 293–322 (1980)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    O’Leary, D.P., White, R.E.: Multi-splittings of matrices and parallel solution of linear systems. SIAM J. Algebraic Discret. Methods 6, 630–640 (1985)CrossRefMATHGoogle Scholar
  26. 26.
    Pearson, J.W., Wathen, A.J.: A new approximation of the Schur complement in preconditioners for PDE-constrained optimization. Linear Algebra Appl. 19, 816–829 (2012)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Pestana, J.: Nonstandard inner products and preconditioned iterative methods. D.Phil. thesis, University of Oxford (2011)Google Scholar
  28. 28.
    Pestana, J., Wathen, A.J.: Combination preconditioning of saddle point systems for positive definiteness. Num. Linear Algebra Appl. 20, 785–808 (2013)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Rees, T., Dollar, H.S., Wathen, A.J.: Optimal solvers for PDE-constrained optimization. SIAM J. Sci. Comput. 32, 271–298 (2010)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Robbé, M., Sadkane, M.: Exact and inexact breakdowns in the block GMRES method. Linear Algebra Appl. 419, 265–285 (2006)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Ruhe, A.: Implementation aspects of band Lanczos algorithms for computation of eigenvalues of large sparse symmetric matrices. Math. Comput. 33, 680–687 (1979)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Rui, P.-L., Yong, H., Chen, R.-S.: Multipreconditioned GMRES method for electromagnetic wave scattering problems. Microw. Opt. Technol. Lett. 50, 150–152 (2008)CrossRefGoogle Scholar
  33. 33.
    Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Comput. 14, 461–469 (1993)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Saad, Y.: Iterative methods for sparse linear systems, 2nd edn. SIAM, Philadelphia (2003)CrossRefMATHGoogle Scholar
  35. 35.
    Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869 (1986)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Sadkane, M.: A block Arnoldi-Chebyshev method for computing the leading eigenpairs of large sparse unsymmetric matrices. Num. Math. 23, 181–193 (2009)MathSciNetMATHGoogle Scholar
  37. 37.
    Sadkane, M.: Block Arnoldi and Davidson methods for unsymmetric large eigenvalue problems. Num. Math. 64, 687–706 (1993)MathSciNetMATHGoogle Scholar
  38. 38.
    Sarkis, M., Szyld, D.B.: Optimal left and right additive Schwarz preconditioning for minimal residual methods with Euclidean and energy norms. Comput. Methods Appl. Mech. Eng. 196, 1612–1621 (2007)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Schöberl, J., Zulehner, W.: Symmetric indefinite preconditioners for saddle point problems with applications to PDE-constrained optimization problems. SIAM J. Matrix Anal. Appl. 29, 752–773 (2007)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Silvester, D., Elman, H., Kay, D., Wathen, A.: Efficient preconditioning of the linearized Navier-Stokes equations for incompressible flow. J. Comput. Appl. Math. 128, 261–279 (2001)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Simoncini, V., Gallopouolos, E.: Convergence properties of block GMRES and matrix polynomials. Linear Algebra Appl. 47, 97–119 (1996)MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Simoncini, V., Szyld, D.B.: On the occurrence of superlinear convergence of exact and inexact Krylov subspace methods. SIAM Rev. 47, 247–272 (2005)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Simoncini, V., Szyld, D.B.: Recent computational developments in Krylov subspace methods for linear systems. Num. Linear Algebra Appl. 14, 1–59 (2007)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Sleijpen, G.L., van der Vorst, H.A., Modersitzki, J.: Differences in the effects of rounding errors in Krylov solvers for symmetric indefinite linear systems. SIAM J. Matrix Anal. Appl. 22, 726–751 (2000)MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Stoll, M., Wathen, A.J.: Combination preconditioning and the Bramble-Pasciak\(^{+}\) preconditioner. SIAM J. Matrix Anal. Appl. 30, 582–608 (2008)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Tröltzsch, F.: Optimal control of partial differential equations: Theory, methods and applications. American Mathematical Society, Providence (2010)CrossRefMATHGoogle Scholar
  47. 47.
    Vital, B.: Etude de quelques méthodes de résolution de problèmes linéaires de grande taille sur multiprocessor. Ph. D. thesis, Université de Rennes (1990)Google Scholar

Copyright information

© Sociedad Española de Matemática Aplicada 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of British ColumbiaVancouverCanada
  2. 2.Department of Scientific ComputingSTFC Rutherford Appleton LaboratoryOxfordshireUK
  3. 3.Department of MathematicsTemple University (038-16)PhiladelphiaUSA

Personalised recommendations