Parallel machines and their communication theoretical limits

  • K. Rüdiger Reischuk
Invited Lectures
Part of the Lecture Notes in Computer Science book series (LNCS, volume 210)

Abstract

We have tried to structure a variety of papers on communication in parallel systems. How much information exchange is necessary for fast computations and how can this be realized on different models of parallel machines? This overview cannot be complete, nor did we try to present any technical details. The interested reader is encouraged to look into the cited literature.

Although we dealt with fast parallel computations the preparation of these notes had to be done in a very short time using an ordinary sequential algorithm. We ask to excuse any deficiencies caused by this fact.

Keywords

Boolean Function Parallel Computation Shared Memory Parallel Machine Input Domain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [B 83]
    M. Ben Or, Lower bounds for algebraic computation trees, Proc. of the 15th ACM An. Symp. on Theory of Computing, 1983, 80–86.Google Scholar
  2. [B 85]
    P. Beame, Lower bounds for very powerfull parallel machines, Technical report, 1985.Google Scholar
  3. [BH 85]
    A. Borodin, J. Hopcroft, Routing, merging and sorting on parallel models of computation, J. Comp. and Syst. Sc. 30, 1985, 130–145.Google Scholar
  4. [CD 82]
    S. Cook, C. Dwork, Bounds on the time for parallel RAMs to compute simple functions, Proc. of the 14th ACM An. Symp. on Theory of Computing, 1982, 231–233.Google Scholar
  5. [CDR 85]
    S. Cook, C. Dwork, R. RelschukUpper and lower time bounds for parallel random access machines without simultaneous writes, SIAM J. Comput. 14, 1985.Google Scholar
  6. [CSV 84]
    A. Chandra, L. Stockmeyer, U. Vishin, Constant depth reducibility, SIAM J. Comput. 13, 1984, 423–439.CrossRefGoogle Scholar
  7. [FMRW 85]
    F. Fich, F. Meyer auf der Helde, P. Ragde, A. Wigderson, One, two, three ... infinity lower bounds for parallel computation, Proc. of the 17th ACM An. Symp. on Theory of Computing, 1985, 48–58.Google Scholar
  8. [FRW 85]
    F. Fich, P. Ragde, A. Wigderson, Relations between concurrent-write models of parallel computation, Technical report, 1985.Google Scholar
  9. [FW 78]
    S. Fortune, J. Wyllie, Parallelism in random access machines, Proc. of the 10th ACM An. Symp. on Theory of Computing, 1978, 114–118.Google Scholar
  10. [G 82]
    L. Goldschlager, A universal interconnection pattern for parallel computers, J. ACM 29, 1982, 1073–1086.Google Scholar
  11. [GL 84]
    A. Gottlieb, C. Kruskal, Complexity results for permuting data and other computations on parallel processors, J. ACM 31, 1984, 193–209.CrossRefGoogle Scholar
  12. [GP 83]
    Z. Galil, W. Paul, An efficient general-purpose parallel computer, J. ACM 30, 1983, 360–387.Google Scholar
  13. [H 85]
    J. Hastad, Improved lower bounds for small depth circuits, Technical report, 1985.Google Scholar
  14. [H 86]
    B. Halstenberg, Kommunikationskomplexität, Diplom Arbeit, Universität Bielefeld, 1986Google Scholar
  15. [HH 81]
    R. Häggkvist, P. Hell, Parallel sorting with constant time for comparisons, SIAM J. Comput. 10, 1981, 465–472.Google Scholar
  16. [JK 84]
    J. Jaja, P. Kumar, Information transfer in distributed computing with application to VLSI, J. ACM 31, 1984, 150–162.Google Scholar
  17. [K 81]
    L. Kucera, Parallel computation and conflicts in memory access, Inf. Proc. Let. 14, 1982, 93–96.Google Scholar
  18. [LPV 81]
    G. Lev, N. Pippenger, L. Vallant, A fast parallel algorithm for routing in permutation networks, IEEE Trans. on Computers 30, 1981, 93–100.Google Scholar
  19. [LYG 85]
    G. Landau, M. Yung, Z. Galil, Distributed algorithms in synchronous broadcasting networks, Proc. of the 12. ICALP, 1985, 363–372.Google Scholar
  20. [MR 84]
    F. Meyer auf der Helde, R. Reischuk, On the limits to speed up parallel machines by large hardware and unbounded communication, Proc. of the 25th IEEE An. Symp. on Foundations of Computer Science, 1984, 56–64.Google Scholar
  21. [MS 82]
    K. Mehlhorn, E. Schmidt, Las Vegas is better than determinism for VLSI circuits, Proc. of the 14th ACM An. Symp. on Theory of Computing, 1982, 330–337.Google Scholar
  22. [MV 84]
    K. Mehlhorn, U. Vishkin, Randomized and deterministic simulation of PRAMS by parallel machines with restricted granularity of parallel memories, Acta Informatica 21, 1984, 339–374.CrossRefGoogle Scholar
  23. [MW 85]
    F. Meyer auf der Heide, A. Wigderson, The complexity of parallel sorting, Proc. of the 26th IEEE An. Symp. on Foundations of Computer Science, 1985, 532–540.Google Scholar
  24. [P 84]
    J. Parberry, A complexity theory of parallel computation, Dissertation, University of Warwick, 1984.Google Scholar
  25. [PV 81]
    F. Preparata, J. Vuillemin, The cube-connected cycles: a versatile network for parallel computation, Com. of the ACM 24, 1981, 300–309.Google Scholar
  26. [R 82]
    R. ReischukA lower time-bound for parallel random access machines without simultaneous writes, IBM Research Report RJ3431, 1982.Google Scholar
  27. [R 85]
    R. Reischuk, Simultaneous writes of parallel random access machines do not help to compute simple arithmetic functions, Technical report, Universität Bielefeld, 1985, to appear in J. ACM.Google Scholar
  28. [R 85a]
    R. Relschuk, An information theoretic lower bound for WRAMS, Technical report, Universität Bielefeld, 1985.Google Scholar
  29. [S 80]
    J. Schwarts, Ultracomputers, ACM Tr. on Progr. Lang. and Systems, 2, 1980, 484–521.Google Scholar
  30. [S 82]
    H.U. Simon, A tight Θ(log log n) — bound on the time for parallel RAM's to compute nondegenerated boolean functions, Information and Control 55, 1982, 102–107.Google Scholar
  31. [S 83]
    A. Schorr, Physical parallel devices are not much faster than sequential ones, Inf. Proc. let. 17, 1983, 103–106.Google Scholar
  32. [S 85]
    M. Snir, On parallel searching, SIAM J. on Computing 14, 1985, 688–708.Google Scholar
  33. [SV 81]
    Y. Shiloach, U. Vishkin, Finding the maximum, merging and sorting in a parallel computation model, J. of Algorithms 2, 1981, 88–102.Google Scholar
  34. [SV 84]
    L. Stockmeyer, U. Vishkin, Simulation of parallel random access machines by circuits, SIAM J. Comput. 13, 1984, 409–422.Google Scholar
  35. [T 84]
    P. Tiwari, Lower bounds on communication complexity in distributed computer networks, Proc. of the 16th ACM An. Symp. on Theory of Computing, 1984, 109–117.Google Scholar
  36. [U 84]
    E. Upfal, Efficient schemes for parallel communication, J. ACM 31, 1984, 507–517.Google Scholar
  37. [U 84a]
    E. Upfal, A probabilistic relation between desirable and feasible models of parallel computation, Proc. of the 16th ACM An. Symp. on Theory of Computing, 1984, 258–265.Google Scholar
  38. [UW 84]
    E. Upfal, A. Wigderson, How to share memory in a distributed system, Proc. of the 25th IEEE An. Symp. on Theory of Computing, 1984, 171–180.Google Scholar
  39. [V 75]
    L. Valiant, Parallelism in comparsion problems, SIAM J. on Computing 4, 1975, 348–355.CrossRefGoogle Scholar
  40. [V 84]
    U. Vishkin, A parallel-design distributed-implementation (PDDI) general-purpose computer, Theoretical Computer Science 32, 1984, 157–172.CrossRefGoogle Scholar
  41. [VW 83]
    U. Vishkin, A. Wigderson, Trade-offs between depth and width in parallel computation, Proc. of the 24th IEEE An. Symp. on Foundations of Computer Science, 1983, 146–153 and SIAM J. on Computing 14, 1985, 303–314.Google Scholar
  42. [Y 79]
    A. Yao, Some complexity questions related to distributive computing, Proc. of the 11th ACM An. Symp. on Theory of Computing, 1979, 209–213.Google Scholar
  43. [Y 82]
    A. Yao, On parallel computation for the knapsack Problem, J. ACM 29, 1982, 898–903.Google Scholar
  44. [Y 85]
    A. Yao, Separating the polynomial-time hierarchy by oracles, Proc. of the 26th An. IEEE Symp. on Foundations of Computer Science, 1985, 1–10.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1985

Authors and Affiliations

  • K. Rüdiger Reischuk
    • 1
  1. 1.Institut für Theoretische InformatikTechnische Hochschule DarmstadtDarmstadtWest-Germany

Personalised recommendations