Advertisement

The Journal of Supercomputing

, Volume 73, Issue 8, pp 3468–3487 | Cite as

JACK: an asynchronous communication kernel library for iterative algorithms

  • Frédéric Magoulès
  • Guillaume Gbikpi-Benissan
Article

Abstract

This article presents a new communication library developed to ease the implementation of both asynchronous and synchronous iterative methods. A mathematical and algorithmic framework about fixed-point methods is described to introduce this class of parallel iterative algorithms, although this library can be used for a larger class of parallel algorithms. After an overview of the main features, we describe detailed implementation aspects arising from the asynchronous context. While the library is mainly based on top of Message Passing Interface library, it has been designed to be easily extended to other types of communication middleware. Finally, some numerical experiments validate this new library, used for implementing both a classical parallel scheme and a sub-structuring approach of the Jacobi iterative method.

Keywords

Asynchronous method Iterative method Sub-structuring method Parallel computing Distributed computing 

References

  1. 1.
    Bahi J, Domas S, Mazouzi K (2004) Jace: a java environment for distributed asynchronous iterative computations. In: Proceedings. 12th euromicro conference on parallel, distributed and network-based processing, 2004, pp 350–357Google Scholar
  2. 2.
    Bahi J, Miellou JC, Rhofir K (1997) Asynchronous multisplitting methods for nonlinear fixed point problems. Numer Algorithms 15(3–4):315–345MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bahi JM, Contassot-Vivier S, Couturier R (2007) Parallel iterative algorithms: from sequential to grid computing. In: Numerical analysis and scientific computing series. Chapman & Hall/CRC, UKGoogle Scholar
  4. 4.
    Bahi JM, Contassot-Vivier S, Couturier R, Vernier F (2005) A decentralized convergence detection algorithm for asynchronous parallel iterative algorithms. IEEE Trans Parallel Distrib Syst 16(1):4–13CrossRefGoogle Scholar
  5. 5.
    Bahi JM, Couturier R, Vuillemin P (2007) Jacev: a programming and execution environment for asynchronous iterative computations on volatile nodes. In: High performance computing for computational science—VECPAR 2006. Lecture Notes in Computer Science, vol 4395. Springer Berlin, pp 79–92 (2007)Google Scholar
  6. 6.
    Baudet GM (1978) Asynchronous iterative methods for multiprocessors. J ACM 25(2):226–244MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bertsekas DP, Tsitsiklis JN (1989) Parallel and distributed computation: numerical methods. Prentice-Hall Inc, Upper Saddle RiverzbMATHGoogle Scholar
  8. 8.
    Bertsekas DP, Tsitsiklis JN (1991) Some aspects of parallel and distributed iterative algorithmsa survey. Automatica 27(1):3–21MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Bru R, Elsner L, Neumann M (1988) Models of parallel chaotic iteration methods. Linear Algebra Appl 103:175–192MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Bru R, Migallon V, Penadés J (1995) Chaotic methods for the parallel solution of linear systems. International meeting on vector and parallel processing. Comput Syst Eng 6(4–5):385–390CrossRefGoogle Scholar
  11. 11.
    Bru R, Migallón V, Penadés J, Szyld DB (1995) Parallel, synchronous and asynchronous two-stage multisplitting methods. Electron Trans Numer Anal 3:24–38MathSciNetzbMATHGoogle Scholar
  12. 12.
    Chajakis ED, Zenios SA (1991) Synchronous and asynchronous implementations of relaxation algorithms for nonlinear network optimization. Parallel Comput 17(8):873–894CrossRefzbMATHGoogle Scholar
  13. 13.
    Chang EJH (1982) Echo algorithms: depth parallel operations on general graphs. IEEE Trans Softw Eng SE–8(4):391–401CrossRefGoogle Scholar
  14. 14.
    Chau M, Couturier R, Bahi J, Spiteri P (2011) Parallel solution of the obstacle problem in grid environments. Int J High Perform Comput Appl 25(4):488–495CrossRefGoogle Scholar
  15. 15.
    Chazan D, Miranker W (1969) Chaotic relaxation. Linear Algebra Appl 2(2):199–222MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Cheik Ahamed A, Magoulès F (2014) Parallel sub-structuring methods for solving sparse linear systems on a cluster of gpus. In: Proceedings of the 16th international conference on high performance and communications (HPCC 2014), Paris, France, Aug. 20–22, 2014. IEEE Computer Society, New York, pp 121–128 (2014)Google Scholar
  17. 17.
    Couturier R, Domas S (2007) Crac: a grid environment to solve scientific applications with asynchronous iterative algorithms. In: Parallel and distributed processing symposium, 2007. IPDPS 2007. IEEE International, New York, pp 1–8Google Scholar
  18. 18.
    Deren W, Zhongzhi B, Evans DJ (1994) Asynchronous multisplitting relaxed iterations for weakly nonlinear systems. Int J Comput Math 54(1–2):57–76CrossRefzbMATHGoogle Scholar
  19. 19.
    Dijkstra EW, Feijen WHJ, van Gasteren AJM (1983) Derivation of a termination detection algorithm for distributed computations. Inf Process Lett 16(5):217–219MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Donnelly JDP (1971) Periodic chaotic relaxation. Linear Algebra Appl 4(2):117–128MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    El Baz D, Spiteri P, Miellou JC, Gazen D (1996) Asynchronous iterative algorithms with flexible communication for nonlinear network flow problems. J Parallel Distrib Comput 38(1):1–15CrossRefGoogle Scholar
  22. 22.
    Evans D, Chikohora S (1998) Convergence testing on a distributed network of processors. Int J Comput Math 70(2):357–378CrossRefzbMATHGoogle Scholar
  23. 23.
    Juliand J, Perrin GR, Spitéri P (1981) Simulations d’exécutions parallèles d’algorithmes numériques asynchrones. In: 1st conférence AMSE, LyonGoogle Scholar
  24. 24.
    Lei L (1989) Convergence of asynchronous iteration with arbitrary splitting form. Linear Algebra Appl 113:119–127MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Lynch NA (1996) Distributed algorithms. Morgan Kaufmann Publishers Inc., San FranciscozbMATHGoogle Scholar
  26. 26.
    Maday Y, Magoulès F (2006) Absorbing interface conditions for domain decomposition methods: a general presentation. Comput Meth Appl Mech Eng 195(29–32):3880–3900MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Magoulès F, Cheik Ahamed A (2015) Alinea: an advanced linear algebra library for massively parallel computations on graphics processing units. Int J High Perform Comput Appl 29(3):284–310CrossRefGoogle Scholar
  28. 28.
    Magoulès F, Venet C (2016) Asynchronous iterative sub-structuring methods. J Math Computer SimulGoogle Scholar
  29. 29.
    Mattern F (1987) Algorithms for distributed termination detection. Distrib Comput 2(3):161–175CrossRefGoogle Scholar
  30. 30.
    OLeary DP, White RE (1985) Multi-splittings of matrices and parallel solution of linear systems. SIAM J Algebraic Discrete Meth 6(4):630–640CrossRefzbMATHGoogle Scholar
  31. 31.
    Perlman R (1985) An algorithm for distributed computation of a spanningtree in an extended lan. SIGCOMM Comput Commun Rev 15(4):44–53CrossRefGoogle Scholar
  32. 32.
    Przemieniecki JS (1963) Matrix structural analysis of substructures. Am Inst Aeronaut Astronaut J 1(1):138–147CrossRefzbMATHGoogle Scholar
  33. 33.
    Rana SP (1983) A distributed solution of the distributed termination problem. Inf Process Lett 17(1):43–46MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Rosenfeld J (1967) A case study on programming for parallel processors. Tech. Rep. RC-64, IBM Thomas J. Watson Research Center, USAGoogle Scholar
  35. 35.
    Savari S, Bertsekas D (1996) Finite termination of asynchronous iterative algorithms. Parallel Comput 22(1):39–56MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Spiteri P, Chau M (2002) Parallel asynchronous Richardson method for the solution of obstacle problem. In: Proceedings. 16th annual international symposium on high performance computing systems and applications, pp 133–138 (2002)Google Scholar
  37. 37.
    Tsitsiklis JN, Bertsekas DP, Athans M (1986) Distributed asynchronous deterministic and stochastic gradient optimization algorithms. IEEE Trans Autom Control 31(9):803–812MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Frédéric Magoulès
    • 1
  • Guillaume Gbikpi-Benissan
    • 2
  1. 1.CentraleSupélecUniversité Paris-SaclayParisFrance
  2. 2.Technological Research Institute SystemXPalaiseauFrance

Personalised recommendations