Definition
Among a set of processing elements (nodes) each node has distinct (personalized) data items destined for each of the other nodes. The all-to-all operation accomplishes this total data exchange among the set of nodes, such that each node ends up having an individual data item from each of the other nodes.
Discussion
The reader may consider first visiting the entry on collective communication.
Let the p nodes be indexed consecutively, 0, 1…, p − 1. Initially each node i has a (column)vector of data x (i) that is further subdivided into subvectors \({x}_{j}^{(i)}\) for \(j = 0,\ldots,p - 1\). The subvector \({x}_{j}^{(i)}\) is to be sent to node j from node i. Upon completion of the all-to-all exchange operation node i will have the vector consisting of the subvectors \({x}_{i}^{(j)}\) for \(j = 0,\ldots,p - 1\). In effect, the matrix consisting of the i columns \({x}^{(i)}\) is transposed with the i...
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Bibliography
Bala V, Bruck J, Cypher R, Elustondo P, Ho A, Ho CT, Kipnis S, Snir M (1995) CCL: a portable and tunable collective communications library for scalable parallel computers. IEEE T Parall Distr 6(2):154–164
Bokhari SH (1996) Multiphase complete exchange: a theoretical analysis. IEEE T Comput 45(2):220–229
Bruck J, Ho CT, Kipnis S, Upfal E, Weathersby D (1997) Efficient algorithms for all-to-all communications in multiport message-passing systems. IEEE T Parall Distr 8(11):1143–1156
Fox G, Johnson M, Lyzenga G, Otto S, Salmon J, Walker D (1988) Solving problems on concurrent processors, vol I. Prentice-Hall, Englewood Cliffs
Goldman A, Peters JG, Trystram D (2006) Exchanging messages of different sizes. J Parallel Distr Com 66(1):1–18
Harary F (1969) Graph theory. Addison-Wesley, Reading, Mass
Johnsson SL, Ho CT (1989) Optimum broadcasting and personalized communication in hypercubes. IEEE T Comput 38(9):1249–1268
Kumar S, Sabharwal Y, Garg R, Heidelberger P (2008) Optimization of all-to-all communication on the blue gene/l supercomputer. In: International conference on parallel processing (ICPP), Portland, pp 320–329
Lam CC, Huang CH, Sadayappan P (1997) Optimal algorithms for all-to-all personalized communication on rings and two dimensional tori. J Parallel Distr Com 43(1):3–13
Liu W, Wang CL, Prasanna VK (2002) Portable and scalable algorithm for irregular all-to-all communication. J Parallel Distr Com 62:1493–1526
Massini A (2003) All-to-all personalized communication on multistage interconnection networks. Discrete Appl Math 128(2–3):435–446
Ranka S, Wang JC, Fox G (1994) Static and run-time algorithms for all-to-many personalized communication on permutation networks. IEEE T Parall Distr 5(12):1266–1274
Ranka S, Wang JC, Kumar M (1995) Irregular personalized communication on distributed memory machines. J Parallel Distr Com 25(1):58–71
Ritzdorf H, Träff JL (2006) Collective operations in NEC’s high-performance MPI libraries. In: International parallel and distributed processing symposium (IPDPS 2006), p 100
Saad Y, Schultz MH (1989) Data communication in parallel architectures. Parallel Comput 11(2):131–150
Scott DS (1991) Efficient all-to-all communication patterns in hypercube and mesh topologies. In: Proceedings 6th conference on distributed memory concurrent Computers, pp 398–403
Suh YJ, Shin KG (2001) All-to-all personalized communication in multidimensional torus and mesh networks. IEEE T Parall Distr 12(1):38–59
Suh YJ, Yalamanchili S (1998) All-to-all communication with minimum start-up costs in 2D/3D tori and meshes. IEEE T Parall Distr 9(5):442–458
Thakur R, Gropp WD, Rabenseifner R (2004) Improving the performance of collective operations in MPICH. Int J High Perform C 19:49–66
Tseng YC, Gupta SKS (1996) All-to-all personalized communication in a wormhole-routed torus. IEEE T Parall Distr 7(5):498–505
Tseng YC, Lin TH, Gupta SKS, Panda DK (1997) Bandwidth-optimal complete exchange on wormhole-routed 2D/3D torus networks: A diagonal-propagation approach. IEEE T Parall Distr 8(4):380–396
Yang Y, Wang J (2000) Optimal all-to-all personalized exchange in self-routable multistage networks. IEEE T Parall Distr 11(3):261–274
Yang Y, Wang J (2002) Near-optimal all-to-all broadcast in multidimensional all-port meshes and tori. IEEE T Parall Distr 13(2):128–141
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this entry
Cite this entry
Träff, J.L., vande Geijn, R.A. (2011). All-to-All. In: Padua, D. (eds) Encyclopedia of Parallel Computing. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-09766-4_34
Download citation
DOI: https://doi.org/10.1007/978-0-387-09766-4_34
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-09765-7
Online ISBN: 978-0-387-09766-4
eBook Packages: Computer ScienceReference Module Computer Science and Engineering