Skip to main content
SpringerLink
Log in

On restarted and deflated block FOM and GMRES methods for sequences of shifted linear systems

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The problem of shifted linear systems is an important and challenging issue in a number of research applications. Krylov subspace methods are effective techniques for different kinds of this problem due to their advantages in large and sparse matrix problems. In this paper, two new block projection methods based on respectively block FOM and block GMRES are introduced for solving sequences of shifted linear systems. We first express the original problem explicitly by a sequence of Sylvester matrix equations whose coefficient matrices are obtained from the shifted linear systems. Then, we show the restarted shifted block FOM (rsh-BFOM) method and derive some of its properties. We also present a framework for the restarted shifted block GMRES (rsh-BGMRES) method. In this regard, we describe two variants of rsh-BGMRES, including (1) rsh-BGMRES with an unshifted base system that applies a fixed unshifted base system and (2) rsh-BGMRES with a variable shifted base system in which the base block system can change after restart. Furthermore, we consider the use of deflation techniques for improving the performance of the rsh-BFOM and rsh-BGMRES methods. Finally, some numerical experiments are conducted to demonstrate the effectiveness of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. When block vectors are used, some authors prefer to use the word cospatial instead of collinear [42, 43]

References

  1. Datta, B.N.: Numerical Methods for Linear Control Systems. Academic Press, 82 (2004)

  2. Laub, A.: Numerical linear algebra aspects of control design computations. IEEE Trans. Autom. Control 30(2), 97–108 (1985). https://doi.org/10.1109/TAC.1985.1103900

    Article  MathSciNet  MATH  Google Scholar 

  3. Bloch, J., Frommer, A., Lang, B., Wettig, T.: An iterative method to compute the sign function of a non-Hermitian matrix and its application to the overlap Dirac operator at nonzero chemical potential. Comput. Phys. Commun. 177(12), 933–943 (2007). https://doi.org/10.1016/j.cpc.2007.07.012, http://www.sciencedirect.com/science/article/pii/S0010465507003633

    Article  MathSciNet  Google Scholar 

  4. Darnell, D., Morgan, R.B., Wilcox, W.: Deflation of eigenvalues for iterative methods in lattice QCD. Nuclear Physics B - Proceedings Supplements 129-130, 856–858 (2004). https://doi.org/10.1016/S0920-5632(03)02734-8, http://www.sciencedirect.com/science/article/pii/S0920563203027348

    Article  Google Scholar 

  5. Freund, R.W.: Solution of shifted linear systems by quasi-minimal residual iterations. Numerical Linear Algebra. Berlin, Boston: De Gruyter, pp 101–121 https://doi.org/10.1515/9783110857658.101 (1993)

  6. Narayanan, R., Neuberger, H.: An Alternative to domain wall fermions. Phys. Rev. D62, 074504 (2000). https://doi.org/10.1103/PhysRevD.62.074504. hep-lat/0005004

    Article  Google Scholar 

  7. Sakurai, T., Tadano, H., Kuramashi, Y.: Application of block Krylov subspace algorithms to the Wilson-Dirac equation with multiple right-hand sides in lattice QCD. Comput. Phys. Commun. 181(1), 113–117 (2010). https://doi.org/10.1016/j.cpc.2009.09.006, http://www.sciencedirect.com/science/article/pii/S0010465509002859

    Article  MathSciNet  Google Scholar 

  8. Gallopoulos, E., Saad, Y.: Efficient solution of parabolic equations by Krylov approximation methods. SIAM J. Sci. Stat. Comput. 13(5), 1236–1264 (1992). https://doi.org/10.1137/0913071

    Article  MathSciNet  MATH  Google Scholar 

  9. Garrappa, R., Popolizio, M.: On the use of matrix functions for fractional partial differential equations. Math. Comput. Simul. 81(5), 1045–1056 (2011). https://doi.org/10.1016/j.matcom.2010.10.009, http://www.sciencedirect.com/science/article/pii/S0378475410003150, Important aspects on structural dynamical systems and their numerical computation

    Article  MathSciNet  Google Scholar 

  10. Meerbergen, K.: The solution of parametrized symmetric linear systems. SIAM Journal on Matrix Analysis and Applications 24(4), 1038–1059 (2003). https://doi.org/10.1137/S0895479800380386

    Article  MathSciNet  MATH  Google Scholar 

  11. Weideman, J.A.C., Trefethen, L.N.: Parabolic and Hyperbolic Contours for computing the Bromwich integral. Math. Comput. 76(259), 1341–1356 (2007). http://www.jstor.org/stable/40234432

    Article  MathSciNet  Google Scholar 

  12. Ikegami, T., Sakurai, T.: Contour integral eigensolver for non-Hermitian systems: A Rayleigh-Ritz-type approach. Taiwanese J. Math. 14(3A), 825–837 (2010). https://doi.org/10.11650/twjm/1500405869. https://doi.org/10.11650/twjm/1500405869

    Article  MathSciNet  MATH  Google Scholar 

  13. Ikegami, T., Sakurai, T., Nagashima, U.: A filter diagonalization for generalized eigenvalue problems based on the SakuraiSugiura projection method. J. Comput. Appl. Math. 233(8), 1927–1936 (2010)

    Article  MathSciNet  Google Scholar 

  14. Feriani, A., Perotti, F., Simoncini, V.: Iterative system solvers for the frequency analysis of linear mechanical systems. Comput. Methods Appl. Mech. Eng. 190(13), 1719–1739 (2000). https://doi.org/10.1016/S0045-7825(00)00187-0. http://www.sciencedirect.com/science/article/pii/S0045782500001870

    Article  MATH  Google Scholar 

  15. Saibaba, A., Bakhos, T., Kitanidis, P.: A flexible Krylov solver for shifted systems with application to Oscillatory Hydraulic Tomography. SIAM J. Sci. Comput. 35(6), A3001–A3023 (2013). https://doi.org/10.1137/120902690

    Article  MathSciNet  MATH  Google Scholar 

  16. Sogabe, T., Hoshi, T., L., Z S, Fujiwara, T.: On a weighted quasi-residual minimization strategy for solving complex symmetric shifted linear systemss. Electron. Trans. Numer. Anal. 31, 126–140 (2008)

    MathSciNet  MATH  Google Scholar 

  17. Sogabe, T., Hoshi, T., Zhang, S.-L., Fujiwara, T.: A numerical method for calculating the Green’s function arising from structure theory. In: Kaneda, Y, Kawamura, H, Sasai, M (eds.) Frontiers of Computational Science, Springer Berlin Heidelberg, Berlin, Heidelberg, pp 189–195 (2007)

  18. Takayama, R., Hoshi, T., Sogabe, T., Zhang, S-L, Fujiwara, T.: Linear algebraic calculation of the Green’s function for large-scale electronic structure theory. Phys. Rev. B 73, 165108 (2006). https://doi.org/10.1103/PhysRevB.73.165108. https://link.aps.org/doi/10.1103/PhysRevB.73.165108

    Article  Google Scholar 

  19. Wu, G., Wang, Y., Jin, X.: A preconditioned and shifted GMRES algorithm for the PageRank problem with multiple damping factors. SIAM J. Sci. Comput. 34(5), A2558–A2575 (2012). https://doi.org/10.1137/110834585

    Article  MathSciNet  MATH  Google Scholar 

  20. Elbouyahyaoui, L., Heyouni, M., Jbilou, K., Messaoudi, A.: A block Arnoldi method for the solution of the Sylvester-observer equations. Electron. Trans. Numer. Anal. 47, 18–36 (2017)

    Article  MathSciNet  Google Scholar 

  21. Benzi, M., Bertaccini, D.: Approximate inverse preconditioning for shifted linear systems. BIT Numer. Math. 43(2), 231–244 (2003). https://doi.org/10.1023/A:1026089811044

    Article  MathSciNet  MATH  Google Scholar 

  22. Dehghan, M., Mohammadi-Arani, R.: Generalized product-type methods based on bi-conjugate gradient (GPBiCG) for solving shifted linear systems. Comput. Appl. Math. 36(4), 1591–1606 (2017). https://doi.org/10.1007/s40314-016-0315-y

    Article  MathSciNet  MATH  Google Scholar 

  23. Frommer, A.: BiCGStab(l) for families of shifted linear systems. Computing 70(2), 87–109 (2003). https://doi.org/10.1007/s00607-003-1472-6

    Article  MathSciNet  MATH  Google Scholar 

  24. van den Eshof, J., Sleijpen, G.L.G.: Accurate conjugate gradient methods for families of shifted systems. Appl. Numer. Math. 49(1), 17–37 (2004). https://doi.org/10.1016/j.apnum.2003.11.010, http://www.sciencedirect.com/science/article/pii/S0168927403001934, Numerical Algorithms, Parallelism and Applications

    Article  MathSciNet  Google Scholar 

  25. Frommer, A., Glässner, U: Restarted GMRES for shifted linear systems. SIAM J. Sci. Comput. 19(1), 15–26 (1998). https://doi.org/10.1137/S1064827596304563

    Article  MathSciNet  MATH  Google Scholar 

  26. Jing, Y.-F., Yuan, P., Huang, T.-Z.: A simpler GMRES and its adaptive variant for shifted linear systems. Numerical Linear Algebra with Applications 24(1), e2076 (2017). https://doi.org/10.1002/nla.2076. e2076 nla.2076

    Article  MathSciNet  MATH  Google Scholar 

  27. Simoncini, V.: Restarted full orthogonalization method for shifted linear systems. BIT Numer. Math. 43(2), 459–466 (2003). https://doi.org/10.1023/A:1026000105893

    Article  MathSciNet  MATH  Google Scholar 

  28. Soodhalter, K.M., Szyld, D.B., Xue, F.: Krylov subspace recycling for sequences of shifted linear systems. Appl. Numer. Math. 81, 105–118 (2014). https://doi.org/10.1016/j.apnum.2014.02.006, http://www.sciencedirect.com/science/article/pii/S0168927414000208

    Article  MathSciNet  Google Scholar 

  29. Gu, X.-M., Huang, T.-Z., Carpentieri, B., Imakura, A., Zhang, K., Du, L.: Efficient variants of the CMRH method for solving a sequence of multi-shifted non-Hermitian linear systems simultaneously. arXiv e-prints, arXiv:1611.00288 (2016)

  30. Gu, X.-M., Huang, T.-Z., Yin, G., Carpentieri, B., Wen, C., Du, L.: Restarted Hessenberg method for solving shifted nonsymmetric linear systems. J. Comput. Appl. Math. 331, 166–177 (2018)

    Article  MathSciNet  Google Scholar 

  31. Ahmad, M., Szyld, D., van Gijzen, M.: Preconditioned multishift BiCG for \({\mathscr{H}}_2\)-optimal model reduction. SIAM Journal on Matrix Analysis and Applications 38(2), 401–424 (2017). https://doi.org/10.1137/130914905

    Article  MathSciNet  MATH  Google Scholar 

  32. Soodhalter, K.M.: Two recursive GMRES-type methods for shifted linear systems with general preconditioning. Electron. Trans. Numer. Anal. 451, 499–523 (2016)

    MathSciNet  MATH  Google Scholar 

  33. Darnell, D., Morgan, R.B., Wilcox, W.: Deflated GMRES for systems with multiple shifts and multiple right-hand sides. Linear Algebra Appl. 429(10), 2415–2434 (2008). https://doi.org/10.1016/j.laa.2008.04.019, http://www.sciencedirect.com/science/article/pii/S0024379508001997, Special Issue in honor of Richard S. Varga

    Article  MathSciNet  Google Scholar 

  34. Yin, J-F, Yin, G-J: Restarted full orthogonalization method with deflation for shifted linear systems. Numerical Mathematics: Theory, Methods and Applications 7(3), 399–412 (2014). https://doi.org/10.1017/S1004897900000179

    Article  MathSciNet  MATH  Google Scholar 

  35. Gu, G.: Restarted GMRES augmented with harmonic Ritz vectors for shifted linear systems. Int. J. Comput. Math. 82(7), 837–849 (2005). https://doi.org/10.1080/00207160512331323317

    Article  MathSciNet  MATH  Google Scholar 

  36. Simoncini, V.: On the numerical solution of AX − XB = C. BIT Numer. Math. 36(4), 814–830 (1996). https://doi.org/10.1007/BF01733793

    Article  MathSciNet  MATH  Google Scholar 

  37. El Guennouni, A., Jbilou, K., Riquet, A.J.: Block Krylov subspace methods for solving large Sylvester equations. Numerical Algorithms 29(1), 75–96 (2002). https://doi.org/10.1023/A:1014807923223

    Article  MathSciNet  MATH  Google Scholar 

  38. Robbé, M., Sadkane, M.: A convergence analysis of GMRES and FOM methods for Sylvester equations. Numerical Algorithms 30(1), 71–89 (2002). https://doi.org/10.1023/A:1015615310584

    Article  MathSciNet  MATH  Google Scholar 

  39. Jbilou, K.: Low rank approximate solutions to large Sylvester matrix equations. Appl. Math. Comput. 177(1), 365–376 (2006)

    MathSciNet  MATH  Google Scholar 

  40. Robbé, M., Sadkane, M.: Use of near-breakdowns in the block Arnoldi method for solving large Sylvester equations. Appl. Numer. Math. 58 (4), 486–498 (2008). https://doi.org/10.1016/j.apnum.2007.01.025, http://www.sciencedirect.com/science/article/pii/S0168927407000372, Selected Papers from the Seventh IMACS International Symposium on Iterative Methods in Scientific Computing

    Article  MathSciNet  Google Scholar 

  41. Datta, B.N., Saad, Y.: Arnoldi methods for large Sylvester-like observer matrix equations, and an associated algorithm for partial spectrum assignment. Linear Algebra Appl. 154-156, 225–244 (1991). https://doi.org/10.1016/0024-3795(91)90378-A, http://www.sciencedirect.com/science/article/pii/002437959190378A

    Article  MathSciNet  MATH  Google Scholar 

  42. Frommer, A., Lund, K., Szyld, D.B.: Block Krylov subspace methods for functions of matrices. Electron. Trans. Numer. Anal. 47, 100–126 (2017)

    MathSciNet  MATH  Google Scholar 

  43. Kubínová, M., Soodhalter, K.M.: Admissible and attainable convergence behavior of block Arnoldi and GMRES. https://arxiv.org/abs/1907.03677 (2019)

  44. Soodhalter, K.: Block Krylov subspace recycling for shifted systems with unrelated right-hand sides. SIAM J. Sci. Comput. 38(1), A302–A324 (2016). https://doi.org/10.1137/140998214

    Article  MathSciNet  Google Scholar 

  45. Sun, D.-L., Huang, T.-Z., Jing, Y.-F., Carpentieri, B.: A block GMRES method with deflated restarting for solving linear systems with multiple shifts and multiple right-hand sides. Numerical Linear Algebra with Applications 25(5), e2148 (2018)

    Article  MathSciNet  Google Scholar 

  46. Wu, G., Pang, H.-K., Sun, J.-L.: A shifted block FOM algorithm with deflated restarting for matrix exponential computations. Appl. Numer. Math. 127, 306–323 (2018). https://doi.org/10.1016/j.apnum.2018.01.015, http://www.sciencedirect.com/science/article/pii/S0168927418300242

    Article  MathSciNet  MATH  Google Scholar 

  47. Simoncini, V., Gallopoulos, E.: Convergence properties of block GMRES and matrix polynomials. Linear Algebra Appl. 247, 97–119 (1996). https://doi.org/10.1016/0024-3795(95)00093-3, http://www.sciencedirect.com/science/article/pii/0024379595000933

    Article  MathSciNet  MATH  Google Scholar 

  48. Gaul, A.: Recycling krylov subspace methods for sequences of linear systems analysis and applications. Ph.D. Thesis (2014)

  49. Parks, M.L., de Sturler, E., Mackey, G., Johnson, D.D., Maiti, S.: Recycling krylov subspaces for sequences of linear systems. SIAM J. Sci. Comput. 28(5), 1651–1674 (2006). https://doi.org/10.1137/040607277

    Article  MathSciNet  MATH  Google Scholar 

  50. Morgan, R.: GMRES with deflated restarting. SIAM J. Sci. Comput. 24(1), 20–37 (2002). https://doi.org/10.1137/S1064827599364659

    Article  MathSciNet  MATH  Google Scholar 

  51. Morgan, R.B.: Restarted block-GMRES with deflation of eigenvalues. Appl. Numer. Math. 54(2), 222–236 (2005)

    Article  MathSciNet  Google Scholar 

  52. Gaul, A., Gutknecht, M.H., Liesen, J., Nabben, R.: A framework for deflated and augmented Krylov subspace methods. SIAM Journal on Matrix Analysis and Applications 34(2), 495–518 (2013)

    Article  MathSciNet  Google Scholar 

  53. Giraud, L., Gratton, S., Pinel, X., Vasseur, X.: Flexible GMRES with deflated restarting. SIAM J. Sci. Comput. 32(4), 1858–1878 (2010)

    Article  MathSciNet  Google Scholar 

  54. Meng, J., Zhu, P.-Y., Li, H.-B., Gu, X.-M.: A deflated block flexible GMRES-DR method for linear systems with multiple right-hand sides. Electron. Trans. Numer. Anal. 41, 478–496 (2014)

    MathSciNet  MATH  Google Scholar 

  55. de Sturler, E.: Nested Krylov methods based on GCR. J. Comput. Appl. Math. 67(1), 15–41 (1996)

    Article  MathSciNet  Google Scholar 

  56. Gutknecht, M.H.: Block krylov space methods for linear systems with multiple right-hand sides: An introduction (2007)

  57. Saad, Y.: Iterative methods for sparse linear systems. Society for Industrial and Applied Mathematics, 2nd edition. https://epubs.siam.org/doi/abs/10.1137/1.9780898718003 (2003)

  58. Kent, M.D.: Chebyshev, Krylov, Lanczos: Matrix relationships and computations. Ph.D. Thesis, Stanford University, Stanford, CA, USA. UMI Order No: GAX89-25896 (1989)

  59. Vital, B.: Etude de quelques méthodes de résolution de problémes linéaires de grande taille sur multiprocesseur. Ph.D. Thesis, Université Rennes 1. http://www.sudoc.fr/044024320 (1990)

  60. Golub, G., Nash, S., Van Loan, C.: A Hessenberg-Schur method for the problem AX + XB = C. IEEE Trans. Autom. Control 24 (6), 909–913 (1979). https://doi.org/10.1109/TAC.1979.1102170

    Article  MathSciNet  MATH  Google Scholar 

  61. Chan, T., Wan, W.: Analysis of projection methods for solving linear systems with multiple right-hand sides. SIAM J. Sci. Comput. 18(6), 1698–1721 (1997). https://doi.org/10.1137/S1064827594273067

    Article  MathSciNet  MATH  Google Scholar 

  62. Simoncini, V., Gallopoulos, E.: An iterative method for nonsymmetric systems with multiple right-hand sides. SIAM J. Sci. Comput. 16(4), 917–933 (1995). https://doi.org/10.1137/0916053

    Article  MathSciNet  MATH  Google Scholar 

  63. Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Transactions on Mathematical Software (TOMS) 38(1), 1 (2011)

    MathSciNet  MATH  Google Scholar 

  64. Morgan, R.B.: A restarted GMRES method augmented with eigenvectors. SIAM Journal on Matrix Analysis and Applications 16(4), 1154–1171 (1995)

    Article  MathSciNet  Google Scholar 

  65. Zhong, H.-X., Gu, X.-M.: A flexible and adaptive Simpler GMRES with deflated restarting for shifted linear systems. Computers & Mathematics with Applications 78(3), 997–1007 (2019)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre Régional des Métiers de l’Education et de la Formation, Fes, Morocco

    Lakhdar Elbouyahyaoui

  2. L.M.P.A, Université du Littoral Côte d’Opale, 50 rue F. Buisson BP. 699, F-62228 Calais Cedex, Dunkirk, France

    Mohammed Heyouni

  3. Department of Applied Mathematics, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman, Kerman, Iran

    Azita Tajaddini

  4. Department of Applied Mathematics, Faculty of Sciences and Modern Technologies, Graduate University of Advanced Technology, Kerman, Iran

    Farid Saberi-Movahed

Authors
  1. Lakhdar Elbouyahyaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed Heyouni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Azita Tajaddini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Farid Saberi-Movahed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed Heyouni.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elbouyahyaoui, L., Heyouni, M., Tajaddini, A. et al. On restarted and deflated block FOM and GMRES methods for sequences of shifted linear systems. Numer Algor 87, 1257–1299 (2021). https://doi.org/10.1007/s11075-020-01007-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-020-01007-3

Keywords

Mathematics Subject Classification (2010)

Search

Navigation