Designing Practical Efficient Algorithms for Symmetric Multiprocessors

(Extended Abstract)
  • David R. Helman
  • Joseph JáJá
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1619)


Symmetric multiprocessors (SMPs) dominate the high-end server market and are currently the primary candidate for constructing large scale multiprocessor systems. Yet, the design of efficient parallel algorithms for this platform currently poses several challenges. In this paper, we present a computational model for designing efficient algorithms for symmetric multiprocessors. We then use this model to create efficient solutions to two widely different types of problems - linked list prefix computations and generalized sorting. Our novel algorithm for prefix computations builds upon the sparse ruling set approach of Reid-Miller and Blelloch. Besides being somewhat simpler and requiring nearly half the number of memory accesses, we can bound our complexity with high probability instead of merely on average. Our algorithm for generalized sorting is a modification of our algorithm for sorting by regular sampling on distributed memory architectures. The algorithm is a stable sort which appears to be asymptotically faster than any of the published algorithms for SMPs. Both of our algorithms were implemented in C using POSIX threads and run on four symmetric multiprocessors - the IBM SP-2 (High Node), the HP-Convex Exemplar (S-Class), the DEC AlphaServer, and the Silicon Graphics Power Challenge. We ran our code for each algorithm using a variety of benchmarks which we identified to examine the dependence of our algorithm on memory access patterns. In spite of the fact that the processors must compete for access to main memory, both algorithms still yielded scalable performance up to 16 processors, which was the largest platform available to us. For some problems, our prefix computation algorithm actually matched or exceeded the performance of the standard sequential solution using only a single thread. Similarly, our generalized sorting algorithm always beat the performance of sequential merge sort by at least an order of magnitude, even with a single thread.


Memory Access Parallel Algorithm Main Memory Cache Line Memory Access Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Aggarwal, B. Alpern, A. Chandra, and M. Snir. A Model for Heirarchical Memory. In Proceedings of the 19th Annual ACM Symposium of Theory of Computing, pages 305–314, May 1987.Google Scholar
  2. 2.
    A. Aggarwal, A. Chandra, and M. Snir. Heirarchical Memory with Block Transfer. In Proceedings of the 28th Annual IEEE Symposium on Foundations of Computer Science, pages 204–216, October 1987.Google Scholar
  3. 3.
    A. Aggarwal and J. Vitter. The Input/Output Complexity of Sorting and Related Problems. Communications of the ACM, 31:1116–1127, 1988.CrossRefMathSciNetGoogle Scholar
  4. 4.
    B. Alpern, L. Carter, E. Feig, and T. Selker. The Uniform Memory Hierarchy Model of Compuatation. Algorithmica, 12:72–109, 1994.zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    R. Anderson and G. Miller. Deterministic Parallel List Ranking. In Proceedings Third Aegean Workshop on Computing, AWOC 88, pages 81–90, Corfu, Greece, June/July 1988. Springer-Verlag.Google Scholar
  6. 6.
    G.E. Blelloch, P.B. Gibbons, Y. Matias, and M. Zagha. Accounting for Memory Bank Contention and Delay in High-Bandwidth Multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 8(9):943–958, 1997.CrossRefGoogle Scholar
  7. 7.
    P.B. Gibbons, Y. Matias, and V. Ramachandran. The Queue-Read Queue-Write PRAM Model: Accounting for Contention in Parallel Algorithms. SIAM Journal on Computing, 1997. To appear.Google Scholar
  8. 8.
    D.R. Helman and J. JáJá. Prefix Computations on Symmetric Multiprocessors. Technical Report CS-TR-3915 and UMIACS-TR-98-38, UMIACS and Electrical Engineering, University of Maryland, College Park, MD, July 1998.Google Scholar
  9. 9.
    D.R. Helman and J. JáJá. Sorting on Clusters of SMPs. In Proceedings of the 12th International Parallel Processing Symposium, Orlando, Florida, April 1998.Google Scholar
  10. 10.
    D.R. Helman, J. JáJá, and D.A. Bader. A New Deterministic Parallel Sorting Algorithm With an Experimental Evaluation. ACM Journal of Experimental Algorithmics, 3(4):1–24, 1998.Google Scholar
  11. 11.
    D.E. Knuth. The Art of Computer Programming: Sorting and Searching, volume 3. Addison-Wesley Publishing Company, Reading, MA, 1973.Google Scholar
  12. 12.
    A. LaMarca and R. Ladner. The Influence of Caches on the Performance of Sorting. In roceedings of the Eighth Annual Symposium on Discrete Algorithms, pages 370–377, January 1997.Google Scholar
  13. 13.
    M. Reid-Miller. List Ranking and List Scan on the Cray C90. Journal of the Computer and System Sciences, 53:344–356, 1996.zbMATHCrossRefGoogle Scholar
  14. 14.
    H. Shi and J. Schaeffer. Parallel Sorting by Regular Sampling. Journal of Parallel and Distributed Computing, 14:361–372, 1992.zbMATHCrossRefGoogle Scholar
  15. 15.
    P. Varman, B. Iyer, and S. scheufler. A Multiprocessor Algorithm for Merging Multiple Sorted Lists. In Proceedings of the 1990 International Conference on Parallel Processing, pages 22–26.Google Scholar
  16. 16.
    Uzi Vishkin. Randomized Speed-Ups in Parallel Computation. In Proceedings of the Sixteenth ACM Symposium on Theory of Computing, pages 230–239, Washington, D.C., 1984.Google Scholar
  17. 17.
    J. Vitter and E. Shriver. Algorithms for Parallel Memory I: Two-Level Memories. Algorithmica, 12:110–147, 1994.zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    J.C. Wyllie. The Complexity of Parallel Computations. PhD thesis, Department of Computer Science, Cornell University, Ithica, NY, 1979.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • David R. Helman
    • 1
  • Joseph JáJá
    • 1
  1. 1.Institute for Advanced Computer Studies & Department of Electrical EngineeringUniversity of MarylandCollege Park

Personalised recommendations