Communication complexity of PRAMs

  • Alok Aggarwal
  • Ashok K. Chandra
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 317)


We propose a model for the concurrent read exclusive write PRAM that captures its communication and computational requirements. For this model, we present several results, including the following:

Two n×n matrices can be multiplied in O(n3/p) computation time and O(n2/p2/3) communication delay using p processors (for pn3 / log3/2n). Furthermore, these bounds are optimal for arithmetic on semirings (using +, × only). For sorting and for FFT graphs, it is shown that communication delay of Ω(n log n/(p log(n/p)) is required for pn/ log n. This bound is tight for FFT graphs; it is also shown to be tight for sorting provided pn1−ε for any fixed ε>0.

Given a binary tree, τ, with n leaves and height h, let D opt (τ) denote the minimum communication delay needed to compute τ. It is shown that Ω(log n)≤D opt (τ)≤\(O(\sqrt n )\), and \(\Omega (\sqrt h )\)D opt O(h), all bounds being the best possible. We also present a simple polynomial algorithm that generates a schedule for computing τ with at most 2D opt (τ) delay.

It is shown that the a communication delay-computation time tradeoff given by Papadimitriou and Ullman for a diamond dag can be achieved for essentially two values of the computation time. We also present DAGs that exhibit proper tradeoffs for a substantial range of time.


Computation Time Binary Tree Directed Acyclic Graph Communication Complexity Global Memory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [A80]
    H. Abelson, "Lower Bounds on Information Transfer in Distributed Systems," J. of ACM, Vol. 27, pp. 384–392, 1980.Google Scholar
  2. [AHU74]
    A. V. Aho, J. E. Hopcroft and J. The D. Ullman, "Design and Analysis of Computer Algorithms," Addison Wesley, 1974.Google Scholar
  3. [AUY83]
    A. V. Aho, J. D. Ullman, and M. Yannakakis, "On Notions of Information Transfer in VLSI Circuits," Proc. 15th Annual ACM Symp. on Theory of Computing, pp. 133–139, 1983.Google Scholar
  4. [Co86]
    R. Cole, "Parallel Merge Sort," Proc. 27th Annual IEEE Conf. on Foundations of Computer Science, pp. 511–516, 1986.Google Scholar
  5. [DGS83]
    P. Duris, Z.Galil, and G. Schnitger, "Lower Bounds on Communication Complexity," Proc. 16th Annual ACM Symp. on Theory of Computing, pp. 133–139, 1983.Google Scholar
  6. [HK81]
    J. W. Hong and H. T. Kung, "I/O Complexity: the Red-Blue Pebble Game," Proc. of 13th Annual ACM Symp. on Theory of Computing, pp. 326–333, May 1981.Google Scholar
  7. [JK84]
    J. Ja'Ja' and P. Kumar, "Information Transfer in Distributed Computing with Applications to VLSI," J. of ACM, Vol. 31, pp. 150–162, 1984.Google Scholar
  8. [K70]
    L. R. Kerr, "The Effect of Algebraic Structure on the Computational Complexity of Matrix Multiplications," Ph.D. Thesis, Cornell University, 1970.Google Scholar
  9. [L85]
    F. T. Leighton, "Tight Bounds on the Complexity of Parallel Sorting," IEEE Trans. on Computers, Vol. C-34, No. 3, April 1985.Google Scholar
  10. [PS81]
    C. H. Papadimitriou and M. Sipser, "Communication Complexity," J. of ACM, Vol. 28, pp. 260–268, 1981.Google Scholar
  11. [PU87]
    C. H. Papadimitriou and J. D. Ullman, "A Communication-Time Tradeoff," SIAM J. of Computing, Vol. 16, pp. 639–647, Aug. 1987.Google Scholar
  12. [PY88]
    C. H. Papadimitriou and M. Yannakakis, "Towards an Architecture-Independent Analysis of Parallel Algorithms" 20th Ann. ACM Symp. on Theory of Computing, 1988.Google Scholar
  13. [S88]
    M. Snir, personal communication.Google Scholar
  14. [T79]
    C. D. Thompson, "Area-Time Complexity for VLSI," Proc. 11th Annual ACM Symp. on Theory of Computing, pp. 81–88, 1979.Google Scholar
  15. [T84]
    P. Tiwari, "Lower Bounds on Communication Complexity in Distributed Computer Networks," Proc. 25th Annual IEEE Symp. on Foundations of Computer Science, pp. 109–117, 1984.Google Scholar
  16. [WF81]
    C. L. Wu and T. Y. Feng, "The Universality of the Shuffle-Exchange Network," IEEE Trans. on Computers, Vol. C-30, No. 5, May 1981, 324–332.Google Scholar
  17. [Y79]
    A. C.-C. Yao, "Some Complexity Questions Related to Distributive Computing," Proc. 11th Annual ACM Symp. on Theory of Computing, pp. 209–213, 1979.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1988

Authors and Affiliations

  • Alok Aggarwal
    • 1
  • Ashok K. Chandra
    • 1
  1. 1.IBM Research DivisionThomas J. Watson Research CenterYorktown Heights

Personalised recommendations