Skip to main content
SpringerLink
Log in

Asynchronous Richardson iterations: theory and practice

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

We consider asynchronous versions of the first- and second-order Richardson methods for solving linear systems of equations. These methods depend on parameters whose values are chosen a priori. We explore the parameter values that can be proven to give convergence of the asynchronous methods. This is the first such analysis for asynchronous second-order methods. We find that for the first-order method, the optimal parameter value for the synchronous case also gives an asynchronously convergent method. For the second-order method, the parameter ranges for which we can prove asynchronous convergence do not contain the optimal parameter values for the synchronous iteration. In practice, however, the asynchronous second-order iterations may still converge using the optimal parameter values, or parameter values close to the optimal ones, despite this result. We explore this behavior with a multithreaded parallel implementation of the asynchronous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. See also [7, 25].

  2. Gene Golub in his thesis [14] calls this a method of averaging, following the nomenclature used by von Neumann.

References

  1. Avron, H., Druinsky, A., Gupta, A.: Revisiting asynchronous linear solvers: Provable convergence rate through randomization. J. ACM 62 (6), 51:1–51:27 (2015)

    Article  MathSciNet  Google Scholar 

  2. Baudet, G.M.: Asynchronous iterative methods for multiprocessors. J. ACM 25(2), 226–244 (1978)

    Article  MathSciNet  Google Scholar 

  3. Berman, A., Plemmons, R.J.: Nonnegative Matrices in the Mathematical Sciences, 3rd edn. Academic Press, New York (1979). Reprinted by SIAM, Philadelphia, 1994

    MATH  Google Scholar 

  4. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, NJ (1989)

    MATH  Google Scholar 

  5. Bethune, I., Bull, J.M., Dingle, N.J., Higham, N.J.: Performance analysis of asynchronous Jacobi’s method implemented in MPI, SHMEM and OpenMP. International Journal on High Performance Computing Applications 28 (1), 97–111 (2014)

    Article  Google Scholar 

  6. Chazan, D., Miranker, W.L.: Chaotic relaxation. Linear Algebra Appl. 2, 199–222 (1969)

    Article  MathSciNet  Google Scholar 

  7. Climent, J.J., Perea, C.: Some comparison theorems for weak nonnegative splittings of bounded operators. Linear Algebra and its Applications 275–276, 77–106 (1998)

    Article  MathSciNet  Google Scholar 

  8. Eiermann, M., Niethammer, W.: On the construction of semiiterative methods. SIAM J. Numer. Anal. 20, 1153–1160 (1983)

    Article  MathSciNet  Google Scholar 

  9. Eiermann, M., Niethammer, W., Varga, R.S.: A study of semiiterative methods for nonsymmetric systems of linear equations. Numer. Math. 47, 505–533 (1985)

    Article  MathSciNet  Google Scholar 

  10. Frankel, S.P.: Convergence rates of iterative treatments of partial differential equations. Mathematical Tables and Aids to Computations 4, 65–75 (1950)

    Article  MathSciNet  Google Scholar 

  11. Frommer, A., Szyld, D.B.: On asynchronous iterations. J. Comput. Appl. Math. 123, 201–216 (2000)

    Article  MathSciNet  Google Scholar 

  12. Glusa, C., Boman, E.G., Chow, E., Rajamanickam, S., Szyld, D.B.: Scalable asynchronous domain decomposition solvers. Tech. Rep. 19-10-11, Department of Mathematics, Temple University (2019). Revised April 2020and July 2020. To appear in SIAM Journal on Scientific Computing

  13. Golub, G., Overton, M.: The convergence of inexact Chebyshev and Richardson iterative methods for solving linear systems. Numer. Math. 53(5), 571–594 (1988)

    Article  MathSciNet  Google Scholar 

  14. Golub, G.H.: The use of Chebyshev matrix polynomials in the iterative solution of linear equations compared to the method of successive relaxation. Ph.D. thesis, Department of Mathematics, University of Illinois, Urbana (1959)

  15. Golub, G.H., Varga, R.S.: Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods, Part I. Numer. Math. 3, 147–156 (1961)

    Article  MathSciNet  Google Scholar 

  16. Hook, J., Dingle, N.: Performance analysis of asynchronous parallel Jacobi. Adv. Eng. Softw. 77(3), 831–866 (2018)

    MathSciNet  MATH  Google Scholar 

  17. Magoulès, F., Szyld, D.B., Venet, C.: Asynchronous optimized Schwarz methods with and without overlap. Numer. Math. 137, 199–227 (2017)

    Article  MathSciNet  Google Scholar 

  18. Marek, I., Szyld, D.B.: Comparison theorems for weak splittings of bounded operators. Numer. Math. 58, 387–397 (1990)

    Article  MathSciNet  Google Scholar 

  19. Richardson, L.F.: The approximate arithmetical solution by finite differences of physical problems involving differential equations with an application to the stresses to a masonry dam. Philosophical Transactions of the Royal Society of London, Series A, Mathematical and Physical Sciences 210, 307–357 (1910)

    MATH  Google Scholar 

  20. Saad, Y.: Iterative methods for linear systems of equations: A brief historical journey. In: Brenner, S.C., Shparlinski, I., Shu, C.-W., Szyld, D.B. (eds.) 75 Years of Mathematics of Computation, pp 197–216. American Mathematical Society, Providence (2020)

  21. Spiteri, P.: Parallel asynchronous algorithms: A survey. Adv. Eng. Softw. 149, 102896 (2020)

    Article  Google Scholar 

  22. Varga, R.S.: Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs (1962). Second Edition, revised and expanded. Springer, Berlin, 2000

    MATH  Google Scholar 

  23. Wolfson-Pou, J., Chow, E.: Convergence models and surprising results for the asynchronous Jacobi method. In: 2018 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2018, Vancouver, BC, Canada, May 21-25, 2018, pp 940–949 (2018)

  24. Wolfson-Pou, J., Chow, E.: Modeling the asynchronous Jacobi method without communication delays. Journal of Parallel and Distributed Computing 128, 84–98 (2019)

    Article  Google Scholar 

  25. Woźnicki, Z.I.: Nonnegative splitting theory. Jpn. J. Ind. Appl. Math. 11, 289–342 (1994)

    Article  MathSciNet  Google Scholar 

  26. Yamazaki, I., Chow, E., Bouteiller, A., Dongarra, J.: Performance of asynchronous optimized Schwarz with one-sided communication. Parallel Comput. 86, 66–81 (2019)

    Article  MathSciNet  Google Scholar 

  27. Young, D.M.: Iterative Solution of Large Linear Systems. Academic Press, New York (1971)

    MATH  Google Scholar 

  28. Young, D.M.: Second-degree iterative methods for the solution of large linear systems. Journal of Approximation Theory 5, 137–148 (1972)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

Work on this paper commenced while the three authors were attending a workshop at the Centre International de Rencontres Mathématiques, Luminy, France, in September 2019. The center’s support for such an event is greatly appreciated.

Funding

Work of the first and third authors was supported in part by the U.S. Department of Energy under grants DE-SC-0016564 and DE-SC-0016578.

Author information

Authors and Affiliations

  1. Georgia Institute of Technology, Atlanta, GA, USA

    Edmond Chow

  2. Bergische Universität Wuppertal, Wuppertal, Germany

    Andreas Frommer

  3. Temple University, Philadelphia, PA, USA

    Daniel B. Szyld

Authors
  1. Edmond Chow
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Frommer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel B. Szyld
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edmond Chow.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chow, E., Frommer, A. & Szyld, D.B. Asynchronous Richardson iterations: theory and practice. Numer Algor 87, 1635–1651 (2021). https://doi.org/10.1007/s11075-020-01023-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-020-01023-3

Keywords

Search

Navigation