Advertisement

Compressed Dictionaries: Space Measures, Data Sets, and Experiments

  • Ankur Gupta
  • Wing-Kai Hon
  • Rahul Shah
  • Jeffrey Scott Vitter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4007)

Abstract

In this paper, we present an experimental study of the space-time tradeoffs for the dictionary problem, where we design a data structure to represent set data, which consist of a subset S of n items out of a universe U = {0, 1,...,u – 1} supporting various queries on S. Our primary goal is to reduce the space required for such a dictionary data structure. Many compression schemes have been developed for dictionaries, which fall generally in the categories of combinatorial encodings and data-aware methods and still support queries efficiently. We show that for many (real-world) datasets, data-aware methods lead to a worthwhile compression over combinatorial methods. Additionally, we design a new data-aware building block structure called BSGAP that presents improvements over other data-aware methods.

Keywords

Query Time Kolmogorov Complexity Inverted Index Binary Search Tree Sequential Decode 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Andersson, A., Thorup, M.: Tight(er) worst-case bounds on dynamic searching and priority queues. In: Proceedings of the ACM Symposium on Theory of Computing (2000)Google Scholar
  2. 2.
    Beame, P., Fich, F.: Optimal bounds for the predecessor problem. In: Proceedings of the ACM Symposium on Theory of Computing (1999)Google Scholar
  3. 3.
    Bell, T.C., Moffat, A., Nevill-Manning, C.G., Witten, I.H., Zobel, J.: Data compression in full-text retrieval systems. Journal of the American Society for Information Science 44(9), 508–531 (1993)CrossRefGoogle Scholar
  4. 4.
    Blandford, D., Blelloch, G.: Compact representations of ordered sets. In: Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (2004)Google Scholar
  5. 5.
    Brodnik, A., Munro, I.: Membership in constant time and almost-minimum space. SIAM Journal on Computing 28(5), 1627–1640 (1999)CrossRefMathSciNetMATHGoogle Scholar
  6. 6.
    Elias, P.: Universal codeword sets and representations of the integers. IEEE Transactions on Information Theory IT-21, 194–203 (1975)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Grossi, R., Sadakane, K.: Squeezing succinct data structures into entropy bounds. In: Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (2006)Google Scholar
  8. 8.
    Gupta, A., Hon, W., Shah, R., Vitter, J.: Compressed data structures: Dictionaries and data-aware measures. In: Proceedings of the IEEE Data Compression Conference (2006)Google Scholar
  9. 9.
    Jacobson, G.: Succinct static data structures. Technical Report CMU-CS-89-112, Carnegie-Mellon University (1989)Google Scholar
  10. 10.
    Klein, S.T., Shapira, D.: Searching in compressed dictionaries. In: Proceedings of the IEEE Data Compression Conference (2002)Google Scholar
  11. 11.
    Munro, J.I.: Tables. Foundations of Software Technology and Theoretical Computer Science 16, 37–42 (1996)Google Scholar
  12. 12.
    Raman, R., Raman, V., Rao, S.S.: Succinct indexable dictionaries with applications to encoding k-ary trees and multisets. In: Proceedings of the ACM-SIAM Symposium on Discrete Algorithms, pp. 233–242 (2002)Google Scholar
  13. 13.
    van Emde Boas, P., Kaas, R., Zijlstra, E.: Design and implementation of an efficient priority queue. Math. Systems Theory 10, 99–127 (1977)CrossRefMATHGoogle Scholar
  14. 14.
    Willard, D.E.: New trie data structures which support very fast search operations. Journal of Computer and System Sciences 28(3), 379–394 (1984)CrossRefMathSciNetMATHGoogle Scholar
  15. 15.
    Witten, I.H., Moffat, A., Bell, T.C.: Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ankur Gupta
    • 1
  • Wing-Kai Hon
    • 1
  • Rahul Shah
    • 1
  • Jeffrey Scott Vitter
    • 1
  1. 1.Department of Computer SciencesPurdue UniversityWest LafayetteUSA

Personalised recommendations