# Parallel machines and their communication theoretical limits

Invited Lectures

First Online:

## Abstract

We have tried to structure a variety of papers on communication in parallel systems. How much information exchange is necessary for fast computations and how can this be realized on different models of parallel machines? This overview cannot be complete, nor did we try to present any technical details. The interested reader is encouraged to look into the cited literature.

Although we dealt with fast parallel computations the preparation of these notes had to be done in a very short time using an ordinary sequential algorithm. We ask to excuse any deficiencies caused by this fact.

## Keywords

Boolean Function Parallel Computation Shared Memory Parallel Machine Input Domain
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## Preview

Unable to display preview. Download preview PDF.

## References

- [B 83]
**M. Ben Or**,*Lower bounds for algebraic computation trees*, Proc. of the 15th ACM An. Symp. on Theory of Computing, 1983, 80–86.Google Scholar - [B 85]
- [BH 85]
**A. Borodin, J. Hopcroft**,*Routing, merging and sorting on parallel models of computation*, J. Comp. and Syst. Sc. 30, 1985, 130–145.Google Scholar - [CD 82]
**S. Cook, C. Dwork**,*Bounds on the time for parallel RAMs to compute simple functions*, Proc. of the 14th ACM An. Symp. on Theory of Computing, 1982, 231–233.Google Scholar - [CDR 85]
**S. Cook, C. Dwork, R. Relschuk***Upper and lower time bounds for parallel random access machines without simultaneous writes*, SIAM J. Comput. 14, 1985.Google Scholar - [CSV 84]
**A. Chandra, L. Stockmeyer, U. Vishin**,*Constant depth reducibility*, SIAM J. Comput. 13, 1984, 423–439.CrossRefGoogle Scholar - [FMRW 85]
**F. Fich, F. Meyer auf der Helde, P. Ragde, A. Wigderson**,*One, two, three ... infinity lower bounds for parallel computation*, Proc. of the 17th ACM An. Symp. on Theory of Computing, 1985, 48–58.Google Scholar - [FRW 85]
**F. Fich, P. Ragde, A. Wigderson**,*Relations between concurrent-write models of parallel computation*, Technical report, 1985.Google Scholar - [FW 78]
**S. Fortune, J. Wyllie**,*Parallelism in random access machines*, Proc. of the 10th ACM An. Symp. on Theory of Computing, 1978, 114–118.Google Scholar - [G 82]
**L. Goldschlager**,*A universal interconnection pattern for parallel computers*, J. ACM 29, 1982, 1073–1086.Google Scholar - [GL 84]
**A. Gottlieb, C. Kruskal**,*Complexity results for permuting data and other computations on parallel processors*, J. ACM 31, 1984, 193–209.CrossRefGoogle Scholar - [GP 83]
**Z. Galil, W. Paul**,*An efficient general-purpose parallel computer*, J. ACM 30, 1983, 360–387.Google Scholar - [H 85]
- [H 86]
- [HH 81]
**R. Häggkvist, P. Hell**,*Parallel sorting with constant time for comparisons*, SIAM J. Comput. 10, 1981, 465–472.Google Scholar - [JK 84]
**J. Jaja, P. Kumar**,*Information transfer in distributed computing with application to VLSI*, J. ACM 31, 1984, 150–162.Google Scholar - [K 81]
**L. Kucera**,*Parallel computation and conflicts in memory access*, Inf. Proc. Let. 14, 1982, 93–96.Google Scholar - [LPV 81]
**G. Lev, N. Pippenger, L. Vallant**,*A fast parallel algorithm for routing in permutation networks*, IEEE Trans. on Computers 30, 1981, 93–100.Google Scholar - [LYG 85]
**G. Landau, M. Yung, Z. Galil**,*Distributed algorithms in synchronous broadcasting networks*, Proc. of the 12. ICALP, 1985, 363–372.Google Scholar - [MR 84]
**F. Meyer auf der Helde, R. Reischuk**,*On the limits to speed up parallel machines by large hardware and unbounded communication*, Proc. of the 25th IEEE An. Symp. on Foundations of Computer Science, 1984, 56–64.Google Scholar - [MS 82]
**K. Mehlhorn, E. Schmidt**,*Las Vegas is better than determinism for VLSI circuits*, Proc. of the 14th ACM An. Symp. on Theory of Computing, 1982, 330–337.Google Scholar - [MV 84]
**K. Mehlhorn, U. Vishkin**,*Randomized and deterministic simulation of PRAMS by parallel machines with restricted granularity of parallel memories*, Acta Informatica 21, 1984, 339–374.CrossRefGoogle Scholar - [MW 85]
**F. Meyer auf der Heide, A. Wigderson**,*The complexity of parallel sorting*, Proc. of the 26th IEEE An. Symp. on Foundations of Computer Science, 1985, 532–540.Google Scholar - [P 84]
**J. Parberry**,*A complexity theory of parallel computation*, Dissertation, University of Warwick, 1984.Google Scholar - [PV 81]
**F. Preparata, J. Vuillemin**,*The cube-connected cycles: a versatile network for parallel computation*, Com. of the ACM 24, 1981, 300–309.Google Scholar - [R 82]
**R. Reischuk***A lower time-bound for parallel random access machines without simultaneous writes*, IBM Research Report RJ3431, 1982.Google Scholar - [R 85]
**R. Reischuk**,*Simultaneous writes of parallel random access machines do not help to compute simple arithmetic functions*, Technical report, Universität Bielefeld, 1985, to appear in J. ACM.Google Scholar - [R 85a]
**R. Relschuk**,*An information theoretic lower bound for WRAMS*, Technical report, Universität Bielefeld, 1985.Google Scholar - [S 80]
- [S 82]
**H.U. Simon**,*A tight*Θ(log log*n*) —*bound on the time for parallel RAM's to compute nondegenerated boolean functions*, Information and Control 55, 1982, 102–107.Google Scholar - [S 83]
**A. Schorr**,*Physical parallel devices are not much faster than sequential ones*, Inf. Proc. let. 17, 1983, 103–106.Google Scholar - [S 85]
- [SV 81]
**Y. Shiloach, U. Vishkin**,*Finding the maximum, merging and sorting in a parallel computation model*, J. of Algorithms 2, 1981, 88–102.Google Scholar - [SV 84]
**L. Stockmeyer, U. Vishkin**,*Simulation of parallel random access machines by circuits*, SIAM J. Comput. 13, 1984, 409–422.Google Scholar - [T 84]
**P. Tiwari**,*Lower bounds on communication complexity in distributed computer networks*, Proc. of the 16th ACM An. Symp. on Theory of Computing, 1984, 109–117.Google Scholar - [U 84]
- [U 84a]
**E. Upfal**,*A probabilistic relation between desirable and feasible models of parallel computation*, Proc. of the 16th ACM An. Symp. on Theory of Computing, 1984, 258–265.Google Scholar - [UW 84]
**E. Upfal, A. Wigderson**,*How to share memory in a distributed system*, Proc. of the 25th IEEE An. Symp. on Theory of Computing, 1984, 171–180.Google Scholar - [V 75]
**L. Valiant**,*Parallelism in comparsion problems*, SIAM J. on Computing 4, 1975, 348–355.CrossRefGoogle Scholar - [V 84]
**U. Vishkin**,*A parallel-design distributed-implementation (PDDI) general-purpose computer*, Theoretical Computer Science 32, 1984, 157–172.CrossRefGoogle Scholar - [VW 83]
**U. Vishkin, A. Wigderson**,*Trade-offs between depth and width in parallel computation*, Proc. of the 24th IEEE An. Symp. on Foundations of Computer Science, 1983, 146–153 and SIAM J. on Computing 14, 1985, 303–314.Google Scholar - [Y 79]
**A. Yao**,*Some complexity questions related to distributive computing*, Proc. of the 11th ACM An. Symp. on Theory of Computing, 1979, 209–213.Google Scholar - [Y 82]
- [Y 85]
**A. Yao**,*Separating the polynomial-time hierarchy by oracles*, Proc. of the 26th An. IEEE Symp. on Foundations of Computer Science, 1985, 1–10.Google Scholar

## Copyright information

© Springer-Verlag Berlin Heidelberg 1985