# Designing Practical Efficient Algorithms for Symmetric Multiprocessors

## Abstract

Symmetric multiprocessors (SMPs) dominate the high-end server market and are currently the primary candidate for constructing large scale multiprocessor systems. Yet, the design of efficient parallel algorithms for this platform currently poses several challenges. In this paper, we present a computational model for designing efficient algorithms for symmetric multiprocessors. We then use this model to create efficient solutions to two widely different types of problems - linked list prefix computations and generalized sorting. Our novel algorithm for prefix computations builds upon the sparse ruling set approach of Reid-Miller and Blelloch. Besides being somewhat simpler and requiring nearly half the number of memory accesses, we can bound our complexity *with high probability* instead of merely *on average*. Our algorithm for generalized sorting is a modification of our algorithm for sorting by regular sampling on distributed memory architectures. The algorithm is a stable sort which appears to be asymptotically faster than any of the published algorithms for SMPs. Both of our algorithms were implemented in C using POSIX threads and run on four symmetric multiprocessors - the IBM SP-2 (High Node), the HP-Convex Exemplar (S-Class), the DEC AlphaServer, and the Silicon Graphics Power Challenge. We ran our code for each algorithm using a variety of benchmarks which we identified to examine the dependence of our algorithm on memory access patterns. In spite of the fact that the processors must compete for access to main memory, both algorithms still yielded scalable performance up to 16 processors, which was the largest platform available to us. For some problems, our prefix computation algorithm actually matched or exceeded the performance of the standard sequential solution using only a single thread. Similarly, our generalized sorting algorithm always beat the performance of sequential merge sort by at least an order of magnitude, even with a single thread.

## Keywords

Memory Access Parallel Algorithm Main Memory Cache Line Memory Access Pattern## Preview

Unable to display preview. Download preview PDF.

## References

- 1.A. Aggarwal, B. Alpern, A. Chandra, and M. Snir. A Model for Heirarchical Memory. In
*Proceedings of the 19th Annual ACM Symposium of Theory of Computing*, pages 305–314, May 1987.Google Scholar - 2.A. Aggarwal, A. Chandra, and M. Snir. Heirarchical Memory with Block Transfer. In
*Proceedings of the 28th Annual IEEE Symposium on Foundations of Computer Science*, pages 204–216, October 1987.Google Scholar - 3.A. Aggarwal and J. Vitter. The Input/Output Complexity of Sorting and Related Problems.
*Communications of the ACM*, 31:1116–1127, 1988.CrossRefMathSciNetGoogle Scholar - 4.B. Alpern, L. Carter, E. Feig, and T. Selker. The Uniform Memory Hierarchy Model of Compuatation.
*Algorithmica*, 12:72–109, 1994.zbMATHCrossRefMathSciNetGoogle Scholar - 5.R. Anderson and G. Miller. Deterministic Parallel List Ranking. In
*Proceedings Third Aegean Workshop on Computing, AWOC 88*, pages 81–90, Corfu, Greece, June/July 1988. Springer-Verlag.Google Scholar - 6.G.E. Blelloch, P.B. Gibbons, Y. Matias, and M. Zagha. Accounting for Memory Bank Contention and Delay in High-Bandwidth Multiprocessors.
*IEEE Transactions on Parallel and Distributed Systems*, 8(9):943–958, 1997.CrossRefGoogle Scholar - 7.P.B. Gibbons, Y. Matias, and V. Ramachandran. The Queue-Read Queue-Write PRAM Model: Accounting for Contention in Parallel Algorithms.
*SIAM Journal on Computing*, 1997. To appear.Google Scholar - 8.D.R. Helman and J. JáJá. Prefix Computations on Symmetric Multiprocessors. Technical Report CS-TR-3915 and UMIACS-TR-98-38, UMIACS and Electrical Engineering, University of Maryland, College Park, MD, July 1998.Google Scholar
- 9.D.R. Helman and J. JáJá. Sorting on Clusters of SMPs. In
*Proceedings of the 12th International Parallel Processing Symposium*, Orlando, Florida, April 1998.Google Scholar - 10.D.R. Helman, J. JáJá, and D.A. Bader. A New Deterministic Parallel Sorting Algorithm With an Experimental Evaluation.
*ACM Journal of Experimental Algorithmics*, 3(4):1–24, 1998.Google Scholar - 11.D.E. Knuth.
*The Art of Computer Programming: Sorting and Searching*, volume 3. Addison-Wesley Publishing Company, Reading, MA, 1973.Google Scholar - 12.A. LaMarca and R. Ladner. The Influence of Caches on the Performance of Sorting. In
*roceedings of the Eighth Annual Symposium on Discrete Algorithms*, pages 370–377, January 1997.Google Scholar - 13.M. Reid-Miller. List Ranking and List Scan on the Cray C90.
*Journal of the Computer and System Sciences*, 53:344–356, 1996.zbMATHCrossRefGoogle Scholar - 14.H. Shi and J. Schaeffer. Parallel Sorting by Regular Sampling.
*Journal of Parallel and Distributed Computing*, 14:361–372, 1992.zbMATHCrossRefGoogle Scholar - 15.P. Varman, B. Iyer, and S. scheufler. A Multiprocessor Algorithm for Merging Multiple Sorted Lists. In
*Proceedings of the 1990 International Conference on Parallel Processing*, pages 22–26.Google Scholar - 16.Uzi Vishkin. Randomized Speed-Ups in Parallel Computation. In
*Proceedings of the Sixteenth ACM Symposium on Theory of Computing*, pages 230–239, Washington, D.C., 1984.Google Scholar - 17.J. Vitter and E. Shriver. Algorithms for Parallel Memory I: Two-Level Memories.
*Algorithmica*, 12:110–147, 1994.zbMATHCrossRefMathSciNetGoogle Scholar - 18.J.C. Wyllie.
*The Complexity of Parallel Computations*. PhD thesis, Department of Computer Science, Cornell University, Ithica, NY, 1979.Google Scholar