An Experimental Study of Priority Queues in External Memory

  • Klaus Brengel
  • Andreas Crauser
  • Paolo Ferragina
  • Ulrich Meyer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1668)

Abstract

A priority queue is a data structure that stores a set of items, each one consisting of a tuple which contains some (satellite) information plus a priority value (also called key) drawn from a totally ordered universe. A priority queue supports the following operations on the processed set: access_minimum (returns the item in the set having minimum key), delete_min (returns and deletes the item in the set having the minimum key) and insert (inserts a new item into the set). Priority queues (hereafter PQs) have numerous important applications: combinatorial optimization (e.g. Dijkstra’s shortest path algorithm [7]), time forward processing [5], job scheduling, event simulation and online sorting, just to cite a few. Many PQ implementations currently exist for small data sets fitting into the internal memory of the computer, e.g. k—ary heaps [23], Fibonacci heaps [10], radix heaps [1], and some of them are also publicly available to the programmers (see e.g. the LEDA library [15]). However, in large-scale event simulations or on instances of very large graph problems (as they recently occur in e.g. geographical information systems), the performance of these internal-memory PQs may significantly deteriorate, thus being a bottleneck for the overall application. In fact, as soon as parts of the PQs do not fit entirely into the internal memory of the computer, but reside in its external memory (e.g. in the hard disk), we may observe a heavy paging activity of the external-memory devices because the pattern of memory accesses is not tuned to exhibit any locality of reference. Due to the technological features of current disk systems [17], this situation may determine a slow down of 5 or 6 orders of magnitude in the final performance of each PQ-operation 1. Consequently, it is required to design PQs which take explicitly into account the physical properties of the disk systems in order to achieve efficient I/O-performances that allow these data structures to be plugged successfully in software libraries.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    R. Ahuja and K. Mehlhorn and J.B. Orlin and R.E. Tarjan, ‘Faster Algorithms for the Shortest Path Problem’, Journal of the ACM, 37:213–223, 1990.MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    L. Arge, ‘The Buffer Tree: A new technique for optimal I/O-algorithms’, Workshop on Algorithms and Data Structures, LNCS 955, 334–345, 1995.Google Scholar
  3. 3.
    R. Bayer and E. McCreight, ‘Organization and Maintenance of Large Ordered Indices’, Acta Informatica, 1:173–189, 1972.CrossRefGoogle Scholar
  4. 4.
    G. S. Brodal and J. Katajainen, ‘Worst-case efficient external memory priority queues’, ScandinavianWorkshop on Algorithm Theory, LNCS 1432, 107–117, 1998.CrossRefGoogle Scholar
  5. 5.
    Y. Chiang, M. T. Goodrich, E. F. Grove, R. Tamassia, D. E. Vengroff, and J. S. Vitter, ‘External-memory graph algorithms’, ACM-SIAM Symposium on Discrete Algorithms, 139–149, 1995.Google Scholar
  6. 6.
    A. Crauser and K. Mehlhorn LEDA-SM, extending LEDA to Secondary Memory. Workshop on Algorithmic Engineering, 1999 (this volume).Google Scholar
  7. 7.
    E.W. Dijkstra. ‘A note on two problems in connection with graphs.’ Num. Math., 1:269–271, 1959.MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    M. Farach and P. Ferragina and S. Muthukrishnan. ‘Overcoming the memory bottleneck in Suffix Tree construction’, IEEE Symposium on Foundations of Computer Science, 194–183, 1998.Google Scholar
  9. 9.
    R. Fadel and K.V. Jakobsen and J. Katajainen and J. Teuhola, ‘External heaps combined with effective buffering’, Proc. Computing: The Australasian Theory Symposium, 1997.Google Scholar
  10. 10.
    M.L. Fredman and R.E. Tarjan, ‘Fibonacci heaps and their use in improved network optimization algorithms.’, Journal of the ACM, 34:596–615, 1987.CrossRefMathSciNetGoogle Scholar
  11. 11.
    D. Hutchinson, A. Maheshwari, J. Sack and R. Velicescu, ‘Early experiences in implementing buffer trees’, Workshop on Algorithmic Engineering, 92–103, 1997.Google Scholar
  12. 12.
    ‘IEEE standard 754-1985 for binary floating-point arithmetic’, reprinted in SIGPLAN 22, 1987.Google Scholar
  13. 13.
    V. Kumar and E.J. Schwabe, ‘Improved Algorithms and Data Structures for Solving Graph Problems in External Memory’, IEEE Symposium on Parallel and Distributed Processing, 169–177, 1996.Google Scholar
  14. 14.
    A. LaMarca and R.E. Ladner, ‘The influence of caches on the performances of heaps’, Tech. Report 96-02-03, UW University, 1996. To appear in Journal of Experimental Algorithmics.Google Scholar
  15. 15.
    K. Mehlhorn and S. Näher. LEDA: A platform for combinatorial and geometric computing. Communication of the ACM, 38:96–102, 1995.CrossRefGoogle Scholar
  16. 16.
    D. Naor, C. U. Martel and N. S. Matloff, ‘Performance of Priority Queue Structures in a Virtual Memory Environment’, The Computer Journal, 34(5):428–437, 1991.CrossRefGoogle Scholar
  17. 17.
    C. Ruemmler and J. Wilkes, ‘An introduction to disk drive modeling’, IEEE Computer, 27(3):17–29, 1994.Google Scholar
  18. 18.
    P. Sanders. ‘Fast priority queues for cached memory’, ALENEX’ 99. Workshop on Algorithmic Engineering and Experimentation, LNCS, 1999 (to appear).Google Scholar
  19. 19.
    M. Thorup. ‘On RAM priority queues’, ACM-SIAM Symposium on Discrete Algorithms, 59–67, 1996.Google Scholar
  20. 20.
    D.E. Vengroff and J.S. Vitter. ‘Supporting I/O-efficient scientific computation using TPIE’, IEEE Symposium on Parallel and Distributed Processing, 74–77, 1995.Google Scholar
  21. 21.
    J.S. Vitter and E.A.M. Shriver, ‘Optimal Algorithms for Parallel Memory I:Two-Level Memories’, Algorithmica,12(2-3):110–147, 1994.MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    J. Vitter. External memory algorithms. Invited Tutorial, ACM Symposium on Principles of Database Systems, 1998. Also Invited Paper in European Symposium on Algorithms, 1998.Google Scholar
  23. 23.
    J.W.J. William. Algorithm 232 (heapsort), In Communications of the ACM, 347–348,1964.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Klaus Brengel
    • 1
  • Andreas Crauser
    • 1
  • Paolo Ferragina
    • 2
  • Ulrich Meyer
    • 1
  1. 1.Max-Planck-Institut für InformatikSaarbrückenGermany
  2. 2.Dipartimento di InformaticaUniversità di PisaPisaItaly

Personalised recommendations