Abstract
Theoretical research on parallel algorithms has focused on NC theory. This motivates the development of parallel algorithms that are extremely fast, but possibly wasteful in their use of processors. Such algorithms seem of limited interest for real applications currently run on parallel computers. This paper explores an alternative approach that emphasizes the efficiency of parallel algorithms. We define a complexity class PE of problems that can be solved by parallel algorithms that are efficient (the speedup is proportional to the number of processors used) and polynomially faster than sequential algorithms. Other complexity classes are also defined, in terms of time and efficiency: A class that has a slightly weaker efficiency requirement than PE, and a class that is a natural generalization of NC. We investigate the relationship between various models of parallel computation, using a newly defined concept of efficient simulation. This includes new models that reflect asynchrony and high communication latency in parallel computers. We prove that the class PE is invariant across the shared memory models (PRAM's) and fully connected message passing machines. These results show that our definitions are robust. Many open problems motivated by our approach are listed.
Extended abstract
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
A. Aggarwal and A. K. Chandra, Communication complexity of PRAM's. 15th ICALP (1988).
H. Alt, T. Hagerup, K. Mehlhorn, and F. P. Preparata, Deterministic simulation of idealized parallel computers on more realistic ones. SIAM Journal of Computing 16 (1987) 808–835.
A. V. Aho, J. E. Hopcroft, and J. D. Ullman, The Design and Analysis of Computer Algorithms. Addison-Wesley (1974).
R. J. Anderson and G. L. Miller, Optical communication for pointer based algorithms. Manuscript.
M. Blum, A machine-independent theory of the complexity of recursive functions. JACM 14 (1967) 322–336.
S. A. Cook, C. Dwork, and R. Reischuk, Upper and lower time bounds for parallel random access machines without simultaneous writes. SIAM Journal of Computing 15 (1986) 87–97.
L. Carter and M. Wegman, New hash functions and their use in authentication and set equality. Journal of Computer and System Sciences 22 (1981) 265–279.
P. W. Dymond and W. L. Ruzzo, Parallel random access machines with owned global memory and deterministic context free language recognition. 13th ICALP (1986) 95–104.
S. Fortune and J. Wyllie, Parallelism in random access machines. 10th ACM STOC (1978) 114–118.
A. Gottlieb, R. Grishman, C. P. Kruskal, K. P. McAuliffe, L. Rudolph, and M. Snir, The NYU Ultracomputer — designing an MIMD parallel machine. IEEE Trans. on Computers TC-32 (1983) 175–189.
A. Gottlieb and C. P. Kruskal, Complexity results for permuting data and other computations on parallel processors. JACM 31 (1984) 193–209.
L. Goldschlager, A unified approach to models of synchronous parallel machines. 10th ACM STOC (1978), 89–94.
L. Kucera, Parallel computation and conflicts in memory accesses. IPL 14 (1982) 93–96.
C. P. Kruskal, T. Madej, and L. Rudolph, Parallel prefix on fully connected direct connection machine. ICPP (1986) 278–283.
C. P. Kruskal, Algorithms for replace-add based paracomputers. ICPP (1982) 219-223.
C. P. Kruskal, Searching, merging, and sorting in parallel computation. IEEE Trans. on Computers TC-32 (1983) 942–946.
C. P. Kruskal, L. Rudolph, and M. Snir, Efficient synchronization in multiprocessors with shared memory. 6th PODC (1986) 218–228.
C. P. Kruskal, L. Rudolph, and M. Snir, A complexity theory of efficient parallel algorithms, Technical Report, IBM T. J. Watson Research Center, 1988.
A. K. Karlin and E. Upfal, Parallel hashing — an efficient implementation of shared memory. 18th ACM STOC (1986) 160–168.
R. M. Karp, E. Upfal, and A. Widgerson, Are search and decision problems computationally equivalent? 17th ACM STOC (1985) 464–475.
G. Lev, N. Pippenger, and L. G. Valiant, A fast parallel algorithm for routing in permutation networks. IEEE Trans. on Computers TC-30 (1981) 93–100.
K. Mehlhorn and U. Vishkin, Randomized and deterministic simulations of PRAMs by parallel machines with restricted granularity of parallel memories. Acta Informatica 21 (1984) 339–374.
A. G. Ranade, How to emulate shared memory. 28th FOCS (1987) 185–194.
J. H. Reif, Depth-first search is inherently sequential. IPL 20 (1985) 229–234.
L. Rudolph, Software structures for ultraparallel computing. Ph.D. Thesis, NYU, 1982.
J. Schwartz, Ultracomputers. ACM TOPLAS 2 (1980) 484–521.
M. Snir, On parallel searching. SIAM Journal of Computing 14 (1985) 688–708.
M. Snir, Asynchronous parallel computations. Manuscript.
U. Vishkin, Implementation of simultaneous memory address access in models that forbid it. Journal of Algorithms 4 (1983) 45–50.
J. S. Vitter and R. A. Simons, New classes for parallel complexity: a study of unification and other complete problems for P. IEEE Trans. on Computers TC-35 (1986) 403–418.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1988 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kruskal, C.P., Rudolph, L., Snir, M. (1988). A complexity theory of efficient parallel algorithms. In: Lepistö, T., Salomaa, A. (eds) Automata, Languages and Programming. ICALP 1988. Lecture Notes in Computer Science, vol 317. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-19488-6_126
Download citation
DOI: https://doi.org/10.1007/3-540-19488-6_126
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-19488-0
Online ISBN: 978-3-540-39291-0
eBook Packages: Springer Book Archive