, Volume 12, Issue 2–3, pp 110–147 | Cite as

Algorithms for parallel memory, I: Two-level memories

  • J. S. Vitter
  • E. A. M. Shriver


We provide the first optimal algorithms in terms of the number of input/outputs (I/Os) required between internal memory and multiple secondary storage devices for the problems of sorting, FFT, matrix transposition, standard matrix multiplication, and related problems. Our two-level memory model is new and gives a realistic treatmentof parallel block transfer, in which during a single I/O each of theP secondary storage devices can simultaneously transfer a contiguous block ofB records. The model pertains to a large-scale uniprocessor system or parallel multiprocessor system withP disks. In addition, the sorting, FFT, permutation network, and standard matrix multiplication algorithms are typically optimal in terms of the amount of internal processing time. The difficulty in developing optimal algorithms is to cope with the partitioning of memory intoP separate physical devices. Our algorithms' performances can be significantly better than those obtained by the wellknown but nonoptimal technique of disk striping. Our optimal sorting algorithm is randomized, but practical; the probability of using more than ι times the optimal number of I/Os is exponentially small inl(logl) log(M/B), whereM is the internal memory size.

Key words

I/O, Input/output Disk Secondary memory Sorting Distribution sort FFT Matrix multiplication Transposition Permutation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Aggarwal and J. S. Vitter, The Input/Output Complexity of Sorting and Related Problems,Communications of the ACM 31(9) (September 1988), 1116–1127.CrossRefMathSciNetGoogle Scholar
  2. [2]
    M, Blum, R. W. Floyd, V. Pratt, R. Rivest, and R. E. Tarjan, Time Bounds for Selection,Journal of Computer and System Sciences 7(4) (1973), 448–461.zbMATHCrossRefMathSciNetGoogle Scholar
  3. [3]
    J. L. Carter and M. N. Wegman, Universal Classes of Hash Functions,Journal of Computer and System Sciences 18 (April 1979), 143–154.zbMATHCrossRefMathSciNetGoogle Scholar
  4. [4]
    R. W. Floyd, Permuting Information in Idealized Two-Level Storage, inComplexity of Computer Calculations, R. Miller and J. Thatcher, eds., Plenum, New York, 1972, pp. 105–109.Google Scholar
  5. [5]
    L. Hellerstein, G. A. Gibson, R. M. Karp, R. H. Katz, and D. A. Patterson, Coding Techniques for Handling Failures in Large Disk Arrays,Algorithmica, this issue, pp. 182–208.Google Scholar
  6. [6]
    W. Jilke, Disk Array Mass Storage Systems: The New Opportunity, Amperif Corporation, September 1986.Google Scholar
  7. [7]
    L. Kleinrock,Queueing Systems, Vol. I, Wiley, New York, 1979.Google Scholar
  8. [8]
    D. Knuth,The Art of Computer Programming, Vol. 3, Addison-Wesley, Reading, MA, 1973.Google Scholar
  9. [9]
    F. T. Leighton, Tight Bounds on the Complexity of Parallel Sorting,IEEE Transactions on Computers 34 (April 1985), 344–354.zbMATHCrossRefMathSciNetGoogle Scholar
  10. [10]
    E. E. Lindstrom and J. S. Vitter, The Design and Analysis of BucketSort for Bubble Memory Secondary Storage,IEEE Transactions on Computers 34 (March 1985), 218–233.CrossRefMathSciNetGoogle Scholar
  11. [11]
    N. B. Maginnis, Store More, Spend Less: Mid-Range Options Around,Computerworld, November 16, 1986, p. 71.Google Scholar
  12. [12]
    M. H. Nodine and J. S. Vitter, Large-Scale Sorting in Parallel Memories,Proceedings of the 3rd Annual ACM Symposium on Parallel Algorithms and Architectures, July 1991, pp. 29–39.Google Scholar
  13. [13]
    M. H. Nodine and J. S. Vitter, Deterministic Distribution Sort in Shared and Distributed Memory Multiprocessors,Proceedings of the 5th Annual ACM Symposium on Parallel Algorithms and Architectures, July 1993, pp. 120–129.Google Scholar
  14. [14]
    D. A. Patterson, G. Gibson, and R. H. Katz, A Case for Redundant Arrays of Inexpensive Disks (RAID),Proceedings of the 1988 ACM SIGMOD International Conference on Management of Data, June 1988, pp. 109–116.Google Scholar
  15. [15]
    J. Savage and J. S. Vitter, Parallelism in Space-Time Tradeoffs, inAdvances in Computing Research, Vol. 4, F. P. Preparata, ed., JAI Press, Greenwich, CT, 1987, pp. 117–146.Google Scholar
  16. [16]
    H. S. Stone, Parallel Processing with the Perfect Shuffle,IEEE Transactions on Computers 20 (February 1971), 153–161.zbMATHCrossRefGoogle Scholar
  17. [17]
    University of California, Massive Information Storage, Management, and Use (NSF Institutional Infrastructure Proposal), Technical Report No. UCB/CSD 89/493, University of California at Berkeley, January 1989.Google Scholar
  18. [18]
    J. S. Vitter and Ph. Flajolet, Average-Case Analysis of Algorithms and Data Structures, inHandbook of Theoretical Computer Science, Jan van Leeuwen, ed., North-Holland, Amsterdam, 1990, pp. 431–524.Google Scholar
  19. [19]
    J. S. Vitter and E. A. M. Shriver, Algorithms for Parallel Memory, II: Hierarchical Multilevel Memories,Algorithmica, this issue, pp. 148–169.Google Scholar
  20. [20]
    C. Wu and T. Feng, The Universality of the Shuffle-Exchange Network,IEEE Transactions on Computers 30 (May 1981), 324–332.zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag New York Inc. 1994

Authors and Affiliations

  • J. S. Vitter
    • 1
  • E. A. M. Shriver
    • 2
  1. 1.Department of Computer ScienceDuke UniversityDurhamUSA
  2. 2.Courant InstituteNew York UniversityNew YorkUSA

Personalised recommendations