Skip to main content
SpringerLink
Log in

Mixed-precision conjugate gradient algorithm using the groupwise update strategy

  • Original Paper
  • Published:
Japan Journal of Industrial and Applied Mathematics Aims and scope Submit manuscript

Abstract

The conjugate gradient (CG) method is the most basic iterative solver for large sparse symmetric positive definite linear systems. In finite precision arithmetic, the residual and error norms of the CG method often stagnate owing to rounding errors. The groupwise update is a strategy to reduce the residual gap (the difference between the recursively updated and true residuals) and improve the attainable accuracy of approximations. However, when there is a severe loss of information in updating approximations, it is difficult to sufficiently reduce the true residual and error norms. To overcome this problem, we propose a mixed-precision algorithm of the CG method using the groupwise update strategy. In particular, we perform the underlying CG iterations with the standard double-precision arithmetic and compute the groupwise update with high-precision arithmetic. This approach prevents a loss of information and efficiently avoids stagnation. Numerical experiments using double-double arithmetic demonstrate that the proposed algorithm significantly improves the accuracy of the approximate solutions with a small overhead of computation time. The presented approach can be used in other related solvers as well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abdelfattah, A., Anzt, H., Boman, E.G., et al.: A survey of numerical linear algebra methods utilizing mixed-precision arithmetic. Int. J. High Perform. Comput. Appl. 35(4), 344–369 (2021). https://doi.org/10.1177/10943420211003313

    Article  Google Scholar 

  2. ADVANPIX: multiprecision computing toolbox for MATLAB. https://www.advanpix.com/

  3. Aihara, K., Komeyama, R., Ishiwata, E.: Variants of residual smoothing with a small residual gap. BIT Numer. Math. 59(3), 565–584 (2019). https://doi.org/10.1007/s10543-019-00751-w

    Article  MathSciNet  Google Scholar 

  4. Clark, M.A., Babich, R., Barros, K., Brower, R.C., Rebbi, C.: Solving lattice QCD systems of equations using mixed precision solvers on GPUs. Comput. Phys. Commun. 181(9), 1517–1528 (2010). https://doi.org/10.1016/j.cpc.2010.05.002

    Article  Google Scholar 

  5. Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. 38(1), 1–25 (2011). https://doi.org/10.1145/2049662.2049663

    Article  MathSciNet  Google Scholar 

  6. Dekker, T.J.: A floating-point technique for extending the available precision. Numer. Math. 18(3), 224–242 (1971). https://doi.org/10.1007/BF01397083

    Article  MathSciNet  Google Scholar 

  7. Demmel, J.W.: Applied Numerical Linear Algebra. SIAM, Philadelphia (1997)

    Book  Google Scholar 

  8. Greenbaum, A.: Estimating the attainable accuracy of recursively computed residual methods. SIAM J. Matrix Anal. Appl. 18(3), 535–551 (1997). https://doi.org/10.1137/S0895479895284944

    Article  MathSciNet  Google Scholar 

  9. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952). https://doi.org/10.6028/jres.049.044

    Article  MathSciNet  Google Scholar 

  10. Higham, N.J.: Accuracy and Stability of Numerical Algorithms, 2nd edn. SIAM, Philadelphia (2002)

    Book  Google Scholar 

  11. Higham, N.J., Mary, T.: Mixed precision algorithms in numerical linear algebra. Acta Numer. 31, 347–414 (2022). https://doi.org/10.1017/S0962492922000022

    Article  MathSciNet  Google Scholar 

  12. Jeannerod, C.P., Rump, S.M.: Improved error bounds for inner products in floating-point arithmetic. SIAM J. Matrix Anal. Appl. 34(2), 338–344 (2013). https://doi.org/10.1137/120894488

    Article  MathSciNet  Google Scholar 

  13. Mukunoki, D., Ogita, T.: Performance and energy consumption of accurate and mixed-precision linear algebra kernels on GPUs. J. Comput. Appl. Math. 372, 112701 (2020). https://doi.org/10.1016/j.cam.2019.112701

    Article  MathSciNet  Google Scholar 

  14. Ogita, T., Rump, S.M., Oishi, S.: Accurate sum and dot product. SIAM J. Sci. Comput. 26(6), 1955–1988 (2005). https://doi.org/10.1137/030601818

    Article  MathSciNet  Google Scholar 

  15. Ozaki, K., Ogita, T.: Generation of linear systems with specified solutions for numerical experiments. Reliab. Comput. 25, 148–167 (2017)

    MathSciNet  Google Scholar 

  16. Sleijpen, G.L.G., van der Vorst, H.A.: Reliable updated residuals in hybrid Bi-CG methods. Computing 56(2), 141–163 (1996). https://doi.org/10.1007/BF02309342

    Article  MathSciNet  Google Scholar 

  17. Williams, S., Waterman, A., Patterson, D.: Roofline: an insightful visual performance model for multicore architectures. Commun. ACM 52(4), 65–76 (2009). https://doi.org/10.1145/1498765.1498785

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the reviewers for their careful reading and insightful comments. The authors are most grateful to Dr. Takeshi Ogita, Professor of Waseda University, for offering the CPU environment in Sect. 5. This study was also supported by Grant number JP20H04195 and JP20KK0259 from the Grants-in-Aid for Scientific Research Program (KAKENHI) of the Japan Society for the Promotion of Science (JSPS).

Author information

Authors and Affiliations

  1. Department of Computer Science, Tokyo City University, 1-28-1 Tamazutsumi, Setagaya-ku, Tokyo, 158-8557, Japan

    Kensuke Aihara

  2. Department of Mathematical Sciences, Shibaura Institute of Technology, 307 Fukasaku, Minuma-ku, Saitama-shi, Saitama, 337-8570, Japan

    Katsuhisa Ozaki

  3. RIKEN Center for Computational Science, 7-1-26 Minatojima-minami-machi, Chuo-ku, Kobe-shi, Hyogo, 650-0047, Japan

    Daichi Mukunoki

Authors
  1. Kensuke Aihara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Katsuhisa Ozaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daichi Mukunoki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kensuke Aihara.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest with regard to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aihara, K., Ozaki, K. & Mukunoki, D. Mixed-precision conjugate gradient algorithm using the groupwise update strategy. Japan J. Indust. Appl. Math. 41, 837–855 (2024). https://doi.org/10.1007/s13160-024-00644-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13160-024-00644-8

Keywords

Mathematics Subject Classification

Search

Navigation