A complexity theory of efficient parallel algorithms

  • Clyde P. Kruskal
  • Larry Rudolph
  • Marc Snir
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 317)


Theoretical research on parallel algorithms has focused on NC theory. This motivates the development of parallel algorithms that are extremely fast, but possibly wasteful in their use of processors. Such algorithms seem of limited interest for real applications currently run on parallel computers. This paper explores an alternative approach that emphasizes the efficiency of parallel algorithms. We define a complexity class PE of problems that can be solved by parallel algorithms that are efficient (the speedup is proportional to the number of processors used) and polynomially faster than sequential algorithms. Other complexity classes are also defined, in terms of time and efficiency: A class that has a slightly weaker efficiency requirement than PE, and a class that is a natural generalization of NC. We investigate the relationship between various models of parallel computation, using a newly defined concept of efficient simulation. This includes new models that reflect asynchrony and high communication latency in parallel computers. We prove that the class PE is invariant across the shared memory models (PRAM's) and fully connected message passing machines. These results show that our definitions are robust. Many open problems motivated by our approach are listed.


Parallel Algorithm Shared Memory Sequential Algorithm Weak Model Biconnected Component 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [A2]
    A. Aggarwal and A. K. Chandra, Communication complexity of PRAM's. 15th ICALP (1988).Google Scholar
  2. [A]
    H. Alt, T. Hagerup, K. Mehlhorn, and F. P. Preparata, Deterministic simulation of idealized parallel computers on more realistic ones. SIAM Journal of Computing 16 (1987) 808–835.Google Scholar
  3. [AH]
    A. V. Aho, J. E. Hopcroft, and J. D. Ullman, The Design and Analysis of Computer Algorithms. Addison-Wesley (1974).Google Scholar
  4. [AM]
    R. J. Anderson and G. L. Miller, Optical communication for pointer based algorithms. Manuscript.Google Scholar
  5. [Bl]
    M. Blum, A machine-independent theory of the complexity of recursive functions. JACM 14 (1967) 322–336.Google Scholar
  6. [CD]
    S. A. Cook, C. Dwork, and R. Reischuk, Upper and lower time bounds for parallel random access machines without simultaneous writes. SIAM Journal of Computing 15 (1986) 87–97.Google Scholar
  7. [CW]
    L. Carter and M. Wegman, New hash functions and their use in authentication and set equality. Journal of Computer and System Sciences 22 (1981) 265–279.Google Scholar
  8. [DR]
    P. W. Dymond and W. L. Ruzzo, Parallel random access machines with owned global memory and deterministic context free language recognition. 13th ICALP (1986) 95–104.Google Scholar
  9. [FW]
    S. Fortune and J. Wyllie, Parallelism in random access machines. 10th ACM STOC (1978) 114–118.Google Scholar
  10. [GG]
    A. Gottlieb, R. Grishman, C. P. Kruskal, K. P. McAuliffe, L. Rudolph, and M. Snir, The NYU Ultracomputer — designing an MIMD parallel machine. IEEE Trans. on Computers TC-32 (1983) 175–189.Google Scholar
  11. [GK]
    A. Gottlieb and C. P. Kruskal, Complexity results for permuting data and other computations on parallel processors. JACM 31 (1984) 193–209.Google Scholar
  12. [Go]
    L. Goldschlager, A unified approach to models of synchronous parallel machines. 10th ACM STOC (1978), 89–94.Google Scholar
  13. [K]
    L. Kucera, Parallel computation and conflicts in memory accesses. IPL 14 (1982) 93–96.Google Scholar
  14. [KM]
    C. P. Kruskal, T. Madej, and L. Rudolph, Parallel prefix on fully connected direct connection machine. ICPP (1986) 278–283.Google Scholar
  15. [k1]
    C. P. Kruskal, Algorithms for replace-add based paracomputers. ICPP (1982) 219-223.Google Scholar
  16. [k2]
    C. P. Kruskal, Searching, merging, and sorting in parallel computation. IEEE Trans. on Computers TC-32 (1983) 942–946.Google Scholar
  17. [K1]
    C. P. Kruskal, L. Rudolph, and M. Snir, Efficient synchronization in multiprocessors with shared memory. 6th PODC (1986) 218–228.Google Scholar
  18. [K2]
    C. P. Kruskal, L. Rudolph, and M. Snir, A complexity theory of efficient parallel algorithms, Technical Report, IBM T. J. Watson Research Center, 1988.Google Scholar
  19. [KU]
    A. K. Karlin and E. Upfal, Parallel hashing — an efficient implementation of shared memory. 18th ACM STOC (1986) 160–168.Google Scholar
  20. [KW]
    R. M. Karp, E. Upfal, and A. Widgerson, Are search and decision problems computationally equivalent? 17th ACM STOC (1985) 464–475.Google Scholar
  21. [LP]
    G. Lev, N. Pippenger, and L. G. Valiant, A fast parallel algorithm for routing in permutation networks. IEEE Trans. on Computers TC-30 (1981) 93–100.Google Scholar
  22. [MV]
    K. Mehlhorn and U. Vishkin, Randomized and deterministic simulations of PRAMs by parallel machines with restricted granularity of parallel memories. Acta Informatica 21 (1984) 339–374.Google Scholar
  23. [Ra]
    A. G. Ranade, How to emulate shared memory. 28th FOCS (1987) 185–194.Google Scholar
  24. [Re]
    J. H. Reif, Depth-first search is inherently sequential. IPL 20 (1985) 229–234.Google Scholar
  25. [Ru]
    L. Rudolph, Software structures for ultraparallel computing. Ph.D. Thesis, NYU, 1982.Google Scholar
  26. [Sc]
    J. Schwartz, Ultracomputers. ACM TOPLAS 2 (1980) 484–521.Google Scholar
  27. [S1]
    M. Snir, On parallel searching. SIAM Journal of Computing 14 (1985) 688–708.Google Scholar
  28. [S2]
    M. Snir, Asynchronous parallel computations. Manuscript.Google Scholar
  29. [Vi]
    U. Vishkin, Implementation of simultaneous memory address access in models that forbid it. Journal of Algorithms 4 (1983) 45–50.Google Scholar
  30. [VS]
    J. S. Vitter and R. A. Simons, New classes for parallel complexity: a study of unification and other complete problems for P. IEEE Trans. on Computers TC-35 (1986) 403–418.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1988

Authors and Affiliations

  • Clyde P. Kruskal
    • 1
  • Larry Rudolph
    • 2
  • Marc Snir
    • 3
  1. 1.Computer Science Department, Inst. for Advanced Computer StudiesUniv. of MarylandCollege ParkUSA
  2. 2.Dept. of Computer ScienceThe Hebrew Univ. of JerusalemJerusalemIsrael
  3. 3.IBM T. J. Watson Res. CenterYorktown HeightsUSA

Personalised recommendations