Multistep matrix splitting iteration preconditioning for singular linear systems

Numerical Algorithms volume 75, pages457–475(2017)Cite this article

Abstract

Multistep matrix splitting iterations serve as preconditioning for Krylov subspace methods for solving singular linear systems. The preconditioner is applied to the generalized minimal residual (GMRES) method and the flexible GMRES (FGMRES) method. We present theoretical and practical justifications for using this approach. Numerical experiments show that the multistep generalized shifted splitting (GSS) and Hermitian and skew-Hermitian splitting (HSS) iteration preconditioning are more robust and efficient compared to standard preconditioners for some test problems of large sparse singular linear systems.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Bai, Z.-Z.: Sharp error bounds of some Krylov subspace methods for non-Hermitian linear systems. Appl. Math. Comput. 109, 273–285 (2000). doi:10.1016/S0096-3003(99)00027-2

    MathSciNet  MATH  Google Scholar 

  2. 2.

    Bai, Z.-Z.: Optimal parameters in the HSS-like methods for saddle-point problems. Numer. Linear Algebra Appl. 16, 447–479 (2009). doi:10.1002/nla.626

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Bai, Z.-Z.: On semi-convergence of Hermitian and skew-Hermitian splitting methods for singular linear systems. Computing 89, 171–197 (2010). doi:10.1007/s00607-010-0101-4

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Bai, Z.-Z., Duff, I.S., Wathen, A.J.: A class of incomplete orthogonal factorization methods. I: Methods and theories. BIT 41, 53–70 (2001). doi:10.1023/A:1021913700691

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Bai, Z.-Z., Golub, G.H., Li, C.-K.: Optimal parameter in Hermitian and skew-Hermitian splitting method for certain two-by-two block matrices. SIAM J. Sci. Comput. 28, 583–603 (2006). doi:10.1137/050623644

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Bai, Z.-Z., Golub, G.H., Ng, M.K.: Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems. SIAM J. Matrix Anal. Appl. 24, 603–626 (2003). doi:10.1137/S0895479801395458

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Bai, Z.-Z., Golub, G.H., Ng, M.K.: On inexact Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems. Linear Algebra Appl. 428, 413–440 (2008). doi:10.1016/j.laa.2007.02.018

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Bai, Z.-Z., Golub, G.H., Pan, J.-Y.: Preconditioned Hermitian and skew-Hermitian splitting methods for non-Hermitian positive semidefinite linear systems. Numer. Math. 98, 1–32 (2004). doi:10.1007/s00211-004-0521-1

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Bai, Z.-Z., Yin, J.-F.: Modified incomplete orthogonal factorization methods using Givens rotations. Computing 86, 53–69 (2009). doi:10.1007/s00607-009-0065-4

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    Benzi, M., Szyld, D.B.: Existence and uniqueness of splittings for stationary iterative methods with applications to alternating methods. Numer. Math. 76, 309–321 (1997). doi:10.1007/s002110050265

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Brown, P.N., Walker, H.F.: GMRES on (nearly) singular systems. SIAM J. Matrix Anal. Appl. 18, 37–51 (1997). doi:10.1137/S0895479894262339

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Campbell, S.L., Meyer, C.D.: Generalized Inverses of Linear Transformations. SIAM, Philadelphia (2009). doi:10.1137/1.9780898719048

  13. 13.

    Cao, Y., Li, S., Yao, L.-Q.: A class of generalized shift-splitting preconditioners for nonsymmetric saddle point problems. Appl. Math. Lett. 49, 20–27 (2015). doi:10.1016/j.aml.2015.04.001

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Cao, Y., Miao, S.-X.: On semi-convergence of the generalized shift-splitting iteration method for singular nonsymmetric saddle point problem. Comput. Math. Appl. 71, 1503–1511 (2016). doi:10.1016/j.camwa.2016.02.027

    MathSciNet  Article  Google Scholar 

  15. 15.

    Cao, Z.-H.: Semiconvergence of extrapolated iterative method for singular linear systems. Appl. Math. Comput. 156, 131–136 (2004). doi:10.1016/j.amc.2003.07.007

    MathSciNet  MATH  Google Scholar 

  16. 16.

    Chen, F.: On choices of iteration parameter in HSS method. Appl. Math. Comput. 271, 832–837 (2015). doi:10.1016/j.amc.2015.09.003

    MathSciNet  Google Scholar 

  17. 17.

    Chen, F., Liu, Q.-Q.: On semi-convergence of modified HSS iteration methods. Numer. Algorithms 64, 507–518 (2013). doi:10.1007/s11075-012-9676-5

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Dax, A.: The convergence of linear stationary iterative processes for solving singular unstructured systems of linear equations. SIAM Rev. 32, 611–635 (1990). doi:10.1137/1032122

    MathSciNet  Article  MATH  Google Scholar 

  19. 19.

    DeLong, M.A., Ortega, J.M.: SOR as a preconditioner. Appl. Numer. Math. 18, 431–440 (1995). doi:10.1016/0168-9274(95)00080-E

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    DeLong, M.A., Ortega, J.M.: SOR as a preconditioner II. Appl. Numer. Math. 26, 465–481 (1998). doi:10.1016/S0168-9274(97)00104-9

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Eldén, L., Simoncini, V.: Solving ill-posed linear systems with GMRES and a singular preconditioner. SIAM J. Matrix Anal. Appl. 33, 1369–1394 (2012). doi:10.1137/110832793

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Frankel, S.T.: Convergence rates of iterative treatments of partial differential equations. Math. Comp. 4, 65–75 (1950). doi:10.1090/S0025-5718-1950-0046149-3

    MathSciNet  Article  Google Scholar 

  23. 23.

    Hayami, K., Sugihara, M.: A geometric view of Krylov subspace methods on singular systems. Numer. Linear Algebra Appl. 18, 449–469 (2011). doi:10.1002/nla.737

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    Hensel, K.: Über Potenzreihen von Matrizen. J. Reine Angew. Math. 155, 107–110 (1926). doi:10.1515/crll.1926.155.107. (in German)

    MathSciNet  MATH  Google Scholar 

  25. 25.

    Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Research Nat. Bur. Standards 49, 409–436 (1952). doi:10.6028/jres.049.044

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Huang, Y.-M.: On m-step Hermitian and skew-Hermitian splitting preconditioning methods. J. Engrg. Math. 93, 77–86 (2015). doi:10.1007/s10665-013-9676-z

    MathSciNet  Article  MATH  Google Scholar 

  27. 27.

    Keller, H.B.: On the solution of singular and semidefinite linear systems by iteration. SIAM J. Numer. Anal. 2, 281–290 (1965). doi:10.1137/0702022

    MathSciNet  MATH  Google Scholar 

  28. 28.

    Li, W., Liu, Y.-P., Peng, X.-F.: The generalized HSS method for solving singular linear systems. J. Comput. Appl. Math. 236, 2338–2353 (2012). doi:10.1016/j.cam.2011.11.020

    MathSciNet  Article  MATH  Google Scholar 

  29. 29.

    Meyer, C.D., Plemmons, R.J.: Convergent powers of a matrix with applications to iterative methods for singular linear systems. SIAM J. Numer. Anal. 14, 699–705 (1977). doi:10.1137/0714047

    MathSciNet  Article  MATH  Google Scholar 

  30. 30.

    Morikuni, K., Hayami, K.: Convergence of inner-iteration GMRES methods for rank-deficient least squares problems. SIAM J. Matrix Anal. Appl 36, 225–250 (2015). doi:10.1137/130946009

    MathSciNet  Article  MATH  Google Scholar 

  31. 31.

    Oldenburger, R.: Infinite powers of matrices and characteristic roots. Duke Math. J. 6, 357–361 (1940). doi:10.1215/S0012-7094-40-00627-5

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Paige, C.C., Saunders, M.A: LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Software 8, 43–71 (1982). doi:10.1145/355984.355989

    MathSciNet  Article  MATH  Google Scholar 

  33. 33.

    Rigal, J.L., Gaches, J.: On the compatibility of a given solution with the data of a linear system. J. Assoc. Comput. Mach. 14, 543–548 (1967). doi:10.1145/321406.321416

    MathSciNet  Article  MATH  Google Scholar 

  34. 34.

    Saad, Y.: Preconditioning techniques for nonsymmetric and indefinite linear systems. J. Comput. Appl. Math. 24, 89–105 (1988). doi:10.1016/0377-0427(88)90345-7

    MathSciNet  Article  MATH  Google Scholar 

  35. 35.

    Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Comput. 14, 461–469 (1993). doi:10.1137/0914028

    MathSciNet  Article  MATH  Google Scholar 

  36. 36.

    Schneider, O.: Krylov Subspace Methods and their Generalizations for Solving Singular Linear Operator Equations with Applications to Continuous Time Markov Chains. Ph.D. thesis, Technischen Universität Bergakademie Freiberg. http://nbn-resolving.de/urn:nbn:de:bsz:105-1148840 (2005)

  37. 37.

    Song, Y.-Z: Semiconvergence of extrapolated iterative methods for singular linear systems. J. Comput. Appl. Math. 106, 117–129 (1999). doi:10.1016/S0377-0427(99)00060-6

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    Song, Y.-Z., Wang, L.: On the semiconvergence of extrapolated iterative methods for singular linear systems. Appl. Numer. Math. 44, 401–413 (2003). doi:10.1016/S0168-9274(02)00168-X

    MathSciNet  Article  MATH  Google Scholar 

  39. 39.

    Tanabe, K.: Characterization of linear stationary iterative processes for solving a singular system of linear equations. Numer. Math. 22, 349–359 (1974). doi:10.1007/BF01436918

    MathSciNet  Article  MATH  Google Scholar 

  40. 40.

    Tebbens, J.D., Meurant, G.: Prescribing the behavior of early terminating GMRES and Arnoldi iterations. Numer. Algorithms 65, 69–90 (2014). doi:10.1007/s11075-013-9695-x

    MathSciNet  Article  MATH  Google Scholar 

  41. 41.

    Vuik, C.: New insights in GMRES-like methods with variable preconditioners. J. Comput. Appl. Math. 61, 189–204 (1995). doi:10.1016/0377-0427(94)00067-B

    MathSciNet  Article  MATH  Google Scholar 

  42. 42.

    Wang, L.: Semiconvergence of two-stage iterative methods for singular linear systems. Linear Algebra Appl. 422, 824–838 (2007). doi:10.1016/j.laa.2006.12.009

    MathSciNet  Article  MATH  Google Scholar 

  43. 43.

    Wei, Y.-M., Wu, H.: Convergence properties of Krylov subspace methods for singular linear systems with arbitrary index. J. Comput. Appl. Math. 114, 305–318 (2000). doi:10.1016/S0377-0427(99)90237-6

    MathSciNet  Article  MATH  Google Scholar 

  44. 44.

    Wen, C., Huang, T.-Z., and Wang, C.: Triangular and skew-symmetric splitting method for numerical solutions of Markov chains. Comput. Math. Appl. 62, 4039–4048 (2011). doi:10.1016/j.camwa.2011.09.041

    MathSciNet  Article  MATH  Google Scholar 

  45. 45.

    Wu, S.-L., Li, C.-X.: On semi-convergence of modified HSS method for a class of complex singular linear systems. Appl. Math. Lett. 38, 57–60 (2014). doi:10.1016/j.aml.2014.07.002

    MathSciNet  Article  MATH  Google Scholar 

  46. 46.

    Yang, A.-L., Wu, Y.-J., Xu, Z.-J.: The semi-convergence properties of MHSS method for a class of complex nonsymmetric singular linear systems. Numer. Algorithms 66, 705–719 (2014). doi:10.1007/s11075-013-9755-2

    MathSciNet  Article  MATH  Google Scholar 

  47. 47.

    Young, D.M.: Iterative Methods for Solving Partial Difference Equations of Elliptic Type. Ph.D. thesis, Harvard University. https://www.ma.utexas.edu/CNA/DMY/david_young_thesis.pdf (1950)

  48. 48.

    Yuan, J.-Y.: The Ostrowski-Reich theorem for SOR iterations: extentions to the rank deficient case. Linear Algebra Appl. 315, 189–196 (2000). doi:10.1016/S0024-3795(00)00148-8

    MathSciNet  Article  MATH  Google Scholar 

  49. 49.

    Zhang, N.-M.: A note on preconditioned GMRES for solving singular linear systems. BIT 50, 207–220 (2010). doi:10.1007/s10543-009-0247-7

    MathSciNet  Article  MATH  Google Scholar 

  50. 50.

    Zhang, M.-M., Lu, T.-T., and Wei, Y.-M.: Semi-convergence analysis of Uzawa methods for singular saddle point problems. J. Comput. Appl. Math. 255, 334–345 (2014). doi:10.1016/j.cam.2013.05.015

    MathSciNet  Article  MATH  Google Scholar 

  51. 51.

    Zheng, B., Bai, Z.-Z., Yang, X.: On semi-convergence of parameterized Uzawa methods for singular saddle point problems. Linear Algebra Appl. 431, 808–817 (2009). doi:10.1016/j.laa.2009.03.033

    MathSciNet  Article  MATH  Google Scholar 

Download references

Author information

Affiliations

  1. Division of Information Engineering, Faculty of Engineering, Information and Systems, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8573, Japan

    Keiichi Morikuni

Authors
  1. Keiichi Morikuni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Keiichi Morikuni.

Additional information

This work was supported by the Czech Academy of Sciences under No. M100301201 and JSPS KAKENHI Grant Number 16K17639.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Morikuni, K. Multistep matrix splitting iteration preconditioning for singular linear systems. Numer Algor 75, 457–475 (2017). https://doi.org/10.1007/s11075-017-0330-0

Download citation

Keywords

  • Preconditioning
  • Inner-outer iteration
  • GMRES method
  • Flexible GMRES method
  • Matrix splitting iterations
  • Singular linear system

Mathematics Subject Classification (2010)

  • 65F08
  • 65F10
  • 65F20
  • 65F50